Querying endpoint/processing response with R/httr or R/curl much slower with API running in Docker

When I run my plumber (plumber_1.0.0) API using Docker (Image rstudio/plumber:latest) and query it using R/httr or R/curl it takes >25 seconds and is substantially
slower than querying the same API without running it in Docker. With download.file it's about the same with and without Docker.
There seems to be an issue with processing the response. I am wondering what causes this strange behaviour and how to resolve it. I pasted the times below.

plumber.R

#* @get /test
function( res){

  res$serializer = serializer_tsv()

  res = data.frame(replicate(400, runif(35000, min=0, max=100)))
}

run-plumber.R

#!/usr/bin/env Rscript

library(plumber)
library(readr)

pr("plumber.R") %>% pr_run(port=4000, host="0.0.0.0")

Dockerfile

FROM rstudio/plumber
RUN R -e "install.packages('readr')"

Building and running Docker container

docker build -t mycustomdocker .

docker run --rm -p 4000:4000 -v `pwd`/run-plumber.R:/run-plumber.R -v `pwd`/plumber.R:/plumber.R mycustomdocker /run-plumber.R

Querying API

library(httr)
library(curl)


system.time({
  r = GET("http://127.0.0.1:4000/test")
  warn_for_status(r)
  stop_for_status(r)
  x = content(r)
})


system.time({
  res = curl_fetch_memory("http://127.0.0.1:4000/test")
})


system.time({
  download.file("http://127.0.0.1:4000/test", tempfile())
})

Response header without Docker

> headers(r)
$date
[1] "Fri, 01 Oct 2021 11:57:39 GMT"

$`content-type`
[1] "text/tab-separated-values; charset=UTF-8"

$`content-length`
[1] "254591323"

attr(,"class")
[1] "insensitive" "list"

Response header with Docker

> headers(r)
$date
[1] "Fri, 01 Oct 2021 12:03:04 GMT"

$`content-type`
[1] "text/tab-separated-values; charset=UTF-8"

$`content-encoding`
[1] "gzip"

$`transfer-encoding`
[1] "chunked"

attr(,"class")
[1] "insensitive" "list"

Times without Docker

# httr
user  system elapsed
8.670   1.057  16.746
  
# curl
user  system elapsed
0.161   0.327   7.016

# download.file
user  system elapsed
0.046   0.298   7.406

Times with Docker

# httr
user  system elapsed
9.834   0.888  34.841
  
# curl
user  system elapsed
1.822   0.299  25.734

# download.file
user  system elapsed
0.062   0.310   7.797

Why is your container gzipping the response?

if you are running docker but the docker infra does not have access to the same cpu threads lets say. you will get less performance for threaded workload like readr format_tsv.

I don't know. I use exactly the code I pasted above and the default plumber image. Nothing changed. Does it have to do with plumber installation in the Docker image then?

I measured speed of readr::format_tsv also without Docker image. Same results. The issue above is not because of readr::format_tsv, it's because httr and curl somehow take longer in processing response after it was retrieved from plumber API. I think it can have to do with the response header.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.