How to specify curl options in httr POST?

General
1

In the documentation the usage is for httr::POST is:

POST(url = NULL, config = list(), ..., body = NULL,
     encode = c("multipart", "form", "json", "raw"), handle = NULL)

It indicates that config is a list of configuration options. However, I have seen the value of the config parameter as config = httr::config(ssl_verifypeer = TRUE).

Should the config parameter equal a list or a request object? Or, will either work?

2

While it does seem intuitive that the default value of an argument would indicate what sort of object that argument can take, it's not always quite so straightforward. In this case, the fact that the default value for config is an empty list doesn't necessarily mean that config can be a simple list of configuration options. In httr, a request is just a specifically structured list with the class request added as an attribute. For example:

str(httr::config(verbose = TRUE))
#> List of 7
#>  $ method    : NULL
#>  $ url       : NULL
#>  $ headers   : NULL
#>  $ fields    : NULL
#>  $ options   :List of 1
#>   ..$ verbose: logi TRUE
#>  $ auth_token: NULL
#>  $ output    : NULL
#>  - attr(*, "class")= chr "request"

The part of the documentation that is informative here is the Arguments section, where config is described as:

Additional configuration settings such as http authentication (authenticate), additional headers (add_headers), cookies (set_cookies) etc. See config for full details and list of helpers.

It perhaps could stand to be clearer, but this is basically saying that you need to use one of the helper functions, or the config() function, to supply a correctly structured config argument.

You can further convince yourself of this with a peek at the source code for http::POST (by entering http::POST — no parentheses! — in the R console or typing httr::POST in the RStudio console or editor and pressing F2) :

function (url = NULL, config = list(), ..., body = NULL, encode = c("multipart", 
  "form", "json", "raw"), handle = NULL) 
{
  encode <- match.arg(encode)
  hu <- handle_url(handle, url, ...)
  req <- request_build("POST", hu$url, body_config(body, match.arg(encode)), 
    as.request(config), ...)
  request_perform(req, hu$handle$handle)
}

The config parameter is wrapped in as.request() and passed to request_build(). These functions aren't documented for external use, but we can look at the source code and see what they do.

as.request() just takes a list and adds the request class attribute to it. Meanwhile,request_build() takes a method and a URL as its first two arguments, then receives arbitrary additional arguments. These are basically all combined into a single request, along the way using request_combine(), which assumes that its arguments will have the proper request structure (it tries to access list elements such as method and headers using $ subsetting).

So the additional arguments to request_build() do need to be properly structured request objects, otherwise when they eventually get passed to request_combine() it won't be able to work with them. Therefore, the config parameter to POST() needs to be a properly structured request object, too. The default empty list is just a way of passing "nothing" on into the request-building flow in a way that won't require the kind of special handling that a NULL object would.

2 Likes