Plumber and httpuv performance inconsistencies

General
,
1

I've been investigating the performance of Plumber and the underlying httpuv package, and I noticed a curious pattern. I was wondering if anyone could shed some light on why this might be happening.

I have a toy Plumber server, which looks like this:

library(plumber)

srv <- plumber$new()
srv$handle("GET", "/", function(req, res) {
  list(hello = "world")
})

srv$run(port = 8080)

If I benchmark the response time using wrk (which is a widely-used tool), I get estimates along the lines of ~2ms on this particular machine:

$ Rscript plumber-bench.R &
$ wrk -c 1 -d 10 -t 1 --latency http://localhost:8080
Running 10s test @ http://localhost:8081
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.71ms    1.81ms  46.63ms   99.38%
    Req/Sec   593.63     47.26   656.00     77.00%
  Latency Distribution
     50%    1.54ms
     75%    1.67ms
     90%    1.83ms
     99%    3.10ms
  5910 requests in 10.00s, 859.95KB read
Requests/sec:    590.80
Transfer/sec:     85.97KB

In contrast, I also have a toy httpuv server which looks like this:

library(methods)
library(httpuv)

httpuv::runServer(
  "0.0.0.0", 8080,
  list(call = function(req) {
      list(
          status = 200L,
          headers = list(
              'Content-Type' = 'text/html'
          ),
          body = "{\"hello\":\"world\"}"
      )
  })
)

If I try the same kind of benchmark, I get something bizarrely different: httpuv seems to have a minimum latency of about 40ms:

$ Rscript httpuv-bench.R &
$ wrk -c 1 -d 10 -t 1 --latency http://localhost:8080
Running 10s test @ http://localhost:8080
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    43.83ms    2.88ms  48.10ms   98.68%
    Req/Sec    22.80      4.51    30.00     72.00%
  Latency Distribution
     50%   44.00ms
     75%   44.01ms
     90%   44.04ms
     99%   44.09ms
  228 requests in 10.00s, 18.10KB read
Requests/sec:     22.80
Transfer/sec:      1.81KB

This seems very odd to me. From my reading of the source code, plumber is essentially calling runServer() in exactly the same way as this toy program, but it manages to get much better performance. Does anyone have a clue as to how this is accomplished? Or is my testing just producing some kind of artifact in the case of httpuv?

For reference: I'm using the latest httpuv (1.4.5) and the latest plumber (0.4.6).

1 Like