Plumber API and package structure

Posit Professional products
, ,
1

Hello there, friends! I hope I am posting this in the right category but feel free to move it. I am writing R packages that, among other things, are going to start publishing plumber APIs. The code in the R package will be used to for the APIs, and it makes sense to package both the code and the API together. I'll be publishing these on RStudio Connect, of which I am a huge fan. :heart_eyes:

My question is about best practices for where to keep the various bits and pieces of code if I want a plumber API packaged together with other code. I am picturing something involving API code in /inst and then a deployAPI.R file inside /R that just has basically plumb() in it, much like Dean Attali's guide for making a Shiny app in an R package. Has anyone else experimented with this? What has/has not worked well? Are there best practices documented out there anywhere?

10 Likes
2

Hey Julia! This sounds like an exciting project, and I definitely think you have posted to the right place. Also love hearing the positive feedback on Connect! I cannot say I have followed a pattern like what you describe, but I do have a handful of random thoughts that I will throw into the mix:

  • Docs on plumber and best practices are unfortunately a bit scarce. I am hopeful that will change as time goes on and people contribute, but I think you may be on the bleeding edge a bit here. That said, @Blair09M has done some writing on some more advanced plumber topics that may be of interest if you have not already seen.
  • In particular, if you want to spread out your API over multiple files in the /inst directory, you can look at the pattern vaguely demoed here:
  • Also, some people seem to have started the discussion on this idea here:
  • As you can see, many of the "current best practices" are hidden in plumber GitHub issues :slight_smile:
  • Programmatic deployment is a bit tricky / limited with Connect today. There are some improvements in the works that will hopefully improve this story for you (i.e. if you wanted to redeploy to the same Connect server / content endpoint, deploy as a result of CI, etc.).
  • I think you could probably annotate the functions directly in your package (i.e. interleave plumber comments with roxygen2 comments), but I have no idea how messy that would get... it could very well be a nightmare to maintain. I did want to highlight that it is possible to name your plumber API functions. I prefer this approach:
#* @get /endpoint
myfunc <- function(){ return("blah") }
7 Likes
3

What's rcpp but an API? If I were so ambitious that's where I'd look first for guidance.

4

I wrote some stuff about packaging shiny apps (it's a bit on the long side, sorry) that more or less applies to plumber APIs too.

http://blog.sellorm.com/2018/05/02/packaging-shiny-apps-a-deep-dive/

4 Likes
5

That is different than Dean's approach but also super helpful to see. Thank you!

6

For anybody who comes by in the future and wonders what I ended up doing, I went with an approach like this:

- mypackage
  |- R
    |- utils.R
    |- ...
  |- inst
    |- plumber
      |- api1
        |- plumber.R
      |- api2
        |- plumber.R
  |- DESCRIPTION
  |- ...

It appears to be working well! The APIs themselves are quite simple in this case.

20 Likes
7

Thanks for sharing @julia! For what it's worth, I'll also share where I'm at right now.

I had something similar set up, with the difference that it was two packages' code that I wanted to expose through an API. First I placed a plumber file in each package's inst, and then had a small piece of code that programmatically defined a new plumber router, and mounted the two packages' apis in different paths.

As the api code grew a bit more complex, I got frustrated that it was less straightforward to test than regular package code. Also, as the two apis shared some of the logic of input processing, this resulted in duplicated code. That's why I ended up stripping the api code out of the two original packages, and made a new package solely for their joint api.

Now instead of defining it inside inst, I decided to go with regular package code. Only one function is exported (the one that runs the api), and the rest (endpoint handlers and aux functions) are not, but they are all straightforward to test with testthat. For me there were multiple (two for the business logic + one for the api) packages involved, but the testing benefit stands even if there is only one.

I'm not sure if it's possible to combine this with the "code-comment-definitions" approach, but at least for now I'm defining the apis programmatically. Swagger doesn't currently support that (easily), and @cole mentioned above that Connect's support is also a bit tricky / limited.

So I'm sure this approach is not for everyone, but I thought I'd share it here.

4 Likes
8

Hi all! Thanks for this thread - I found it very useful to pack our production APIs and thanks to @julia for the offline conversation to clarify some points.

We ended up with pretty much the same structure with additional code that doesn't only test the contents of the R folder but actually runs the API itself that sits in inst folder and checks it A to Z for the response validity. Now we can really make sure that things are working as expected before deploying.

Our solution below:

context("API testing")


test_that("local deployment works", {
    
    rs <- callr::r_bg(
    function() {
        pr <- plumber::plumb(file.path("path", "inst", "plumber.R"));
        pr$run(port=8000)
    })
    
    Sys.sleep(3)
    
    link <- "http://localhost:8000/calculate?x=2&y=4"
    
    call <- tryCatch(
    httr::GET(
    url = link,
    httr::add_headers(
    `accept` = 'application/json'),
    httr::content_type("application/json")
    ),
    error = function(x) NA)
    
    print(call)
    
    call_parse <- tryCatch(
    httr::content(call, encoding = "UTF-8"),
    error = function(x) NA)
    
    print(call_parse)
    
    expect_equal(as.numeric(unlist(call_parse)), as.numeric(3))
    
    rs$kill()
})

Brief explanation: we need to run another R process in the background where that particular API would be running so that the main thread could be occupied by the test. We also need to put the execution for a short sleep because it takes a short moment for the API to start. Hope somebody finds this useful!

9 Likes
9

I've created a basic package to demonstrate bundling Plumber APIs inside an R package: https://github.com/sol-eng/plumbpkg

12 Likes
11

Reviving this topic as there is an official solution in plumber v1.0.0 .

@julia's setup was a great inspiration!

- mypackage
  |- R
    |- utils.R
    |- ...
  |- inst
    |- plumber
      |- api1
        |- plumber.R
      |- api2
        |- plumber.R
  |- DESCRIPTION
  |- ...

To plumber::plumb() api1 in mypackage:

plumber::plumb_api("mypackage", "api1")

To see the available APIs for mypackage, call:

# For {mypackage}
plumber::available_apis("mypackage")
#> Available Plumber APIs:
#> * mypackage
#>   - api1
#>   - api2

# For all packages
plumber::available_apis()
#> Available Plumber APIs:
#> * mypackage
#>   - api1
#>   - api2
#> * plumber
#>   - 01-append
#>   - 02-filters
#> ....
9 Likes