We have a problem with a Plumber API which is deployed to Posit Connect (v. 2023.03.0) on an Ubuntu 20.04 virtual machine. The API runs processes which could take longer than 60s to complete (it fetches data from other APIs would could exceed 60s).
Whenever the response time is more than 60s, the API returns a "504 Gateway Time-out - The server didn't respond in time." error. However, the API remains active and even though the 504 error is returned, any processes such as writing data to a Connect pin will be completed after the 504 error message has been returned. The API therefore remains active after the 60s timeout period.
As far as I can establish, there are no settings within Posit Connect (other than that mentioned below) which could result in the timeout occurring. My suspicion is that it is either the virtualisation platform on which Posit Connect is installed, or the firewall of the environment which cause the timeout.
The Runtime Settings for the API on Posit Connect are as following:
- Initial Timeout: 600s (increased from the default of 60s to ensure this is not the cause)
- Connection timeout: 3600s
- Idle Timeout: 120s
- Read timeout: 3600s
I include a reprex below of a Plumber API which also returns a 504 timeout when the delay parameter of the API is more than 60.
library(plumber)
library(pins)
## Configure API
#* @apiTitle Testing Plumber Posit Connect Timeout
#* @apiVersion 0.0.1
#* @apiDescription Reprex to replicate Plumber API 60s timeout
## Configure API endpoint
#* @get /TimeoutTest
#* @param Delay The time in seconds which the API should wait before returning a response.
#* @response default Returns a dataframe in JSON format
TimeoutTest <- function(Delay) {
## Delay API response for the period specified in Delay
StartTime = Sys.time()
Sys.sleep(Delay)
DelayEnd = Sys.time()
## Configure API payload reponse
payload <- paste0("Delay: ", Delay, "s, Start Time: ", StartTime, " - End Time: ", DelayEnd)
## configure and write the payload to a Posit Connect Pin
board <- board_connect(
server = "https://connect.server.com",
auth = "manual",
key= Sys.getenv("CONNECT_API_KEY")
)
pin_write(board = board,
payload,
name = "API_Timeout_Pin",
type = "json")
## return payload as the API GET request response
return(payload)
}
If "Delay" is less than 60, then the API returns a JSON payload and the payloads is also written to the pin on Connect. If "Delay" is more than 60s, a 504 error is returned. However the response is still written to the Pin, implying that the API process remains active even after the 504 is returned.
Any advice on Posit Connect or Plumber API settings which I should verify will be appreciated.
Anyone with advice on firewall configuration settings (in this case a Sophos physical firewall) will also be appreciated. I'd like to pass that on to my IT support team.