Connect API Specification

rstudio-connect
#1

I notice that there's an API specification here:

http://docs.rstudio.com/connect/1.6.2/api/

But it seems to be pretty barebones.

At rstudio::conf I was told that nearly everything is available through the api. But I can't seem to find any reference to any other endpoints.

Has anyone else explored this more?

Ideally I'd like to be able to pull the full list of apps + permissions on demand.
(sitemap.xml doesn't give enough meta information)

I could probably generate this from the audit log but that seems like a lot of munging for what is probably just an undocumented endpoint.

#2

There may have been a mix-up in discussing the Server API at rstudio::conf, or perhaps I am missing something. That is definitely our vision for where the Server API will someday be, but the Server API made its entrance in Connect 1.6.0 (released in April 2018) and is currently in its early days. The Server API endpoints are backwards compatible, public, and documented.

As you have suggested, there is an internal API that is undocumented, but we do not recommend or support trying to use it, make no guarantees about its behavior or backwards compatibility, and ultimately would recommend waiting until we add content listing (the feature that you are requesting) to the Connect Server API.

In the meantime, we would love feedback on the Server API features that we have exposed, if you have any need to utilize them! Stay tuned to the RStudio Blog (blog.rstudio.com) or subscribe to stay up to date on latest releases and the features we continue adding to the Server API.

At the time of rstudio::conf, the Server API did not exist yet, so I expect that discussion was occurring around future vision, may have been discussing the internal API, or there was a mix-up discussing API assets like Plumber REST APIs or Tensorflow models. I for one am super excited about the Server API's future, and am encouraged to hear others excited about additional features.

#3

Thanks for the reply!

Currently an unstable API would be fine.

I'm more interested in 1-time pulls to get at the database without having to go through the backup -> SQLite -> remember the schema.

If I could get a nice list of everything I have, I don't need the endpoint to be supported long term.

Feedback

on the features that are currently documented:
With the exception of a security audit (where I will certainly need to pull everything), it seems as though it's lacking a way to search for particular kinds of audit events.

For example, all successful (or unsuccessful) deployments would be really nice.

#4

Thanks, @shapenaji! That feature (filtering for audit logs) is currently on our backlog. We'll be chipping away at bringing in new API endpoints (and enhancing the existing ones) in the coming months.

#5

Hi Cole,

It seems we can only fetch 500 log-entry with the property $limit. Is there a way we can fetch additional logs? For example I have fetched 500 logs in descending order and stored in list; how can I fetch the next 500 and so on?

#6

You need to page through the results. See the paging docs for the Audit log endpoint here: http://docs.rstudio.com/connect/api/#AuditPager

Also, parameter docs here: http://docs.rstudio.com/connect/api/#get-audit-logs

And cookbook examples here: http://docs.rstudio.com/connect/user/cookbook.html#use-keyset-cursor-pagination

#7

Hi Cole, I think I have figured it out. I was planning to append the log-entry as list and I have used (repeat) loop on next and it seems to be fetching all the required data.

Can I pull application id, application run history, etc from audit log?

#8

Yes, I believe you should be able to pull some of that information. I expect that it will be a decent amount of work to parse the data that you are interested in. I am definitely interested in hearing how you get along, though! If you can make your parsing work / functions open source, I definitely think there would be desire by others to do similar types of exploration. In related fashion, if there are certain tasks that are exceptionally difficult, please let us know, as we are happy to have the feedback / feature requests!

#9

Hi Cole,
I was able to pull app count; but didn't get anything like run history.

#10

Ohh yeah. Sorry, I didn't think about Shiny application start / stops. Yeah, I don't think those are in the Audit logs. You should have render history / etc. of Rmd's and such, though. A good way to test would be to poke around at different things and then see what shows up in the audit logs (that is how I confirmed render history of Rmd reports).

#11

Hi Cole, I have been able to pull users and specific actions from audit logs. The app/code works fine on my desktop, however its fails with error as I publish in connect server.

Error in curl::curl_fetch_memory(url, handle = handle)
SSL certificate problem: self signed certificate in certificate chain

connect is HTTPS server.

#12

It sounds like Connect is using a self-signed certificate in this case? Is that true? If so, you will need to set up the server so that it is possible to type curl https://myconnect.com:port from the Connect server and get a valid response (rather than a self-signed certificate problem).

Again, if this is the case, I suspect you have done some such setup on your desktop prior to publishing to RStudio Connect?

#13

In my desktop: I added certificate path and details as I call rsconnect package and it was able to publish.
I am wondering How I can achieve the same in connect servers.

#14

@Ankur Right, so it sounds like you have a self-signed certificate in use for RStudio Connect, which is why the certificate path / details is needed when you publish to RStudio Connect. You can set up the Connect server itself to trust that certificate, as well, and there are a myriad of ways to do so that depend on your operating system.

Are you the RStudio Connect administrator, or do you have an administrator who can set up the server to trust its own certificate? As I mentioned above, you need to be able to log into the RStudio Connect server via SSH and execute curl https://myconnect.com:port/ without a certificate complaint. Once that is done, the content on Connect should be able to access the API over https without this issue.

A hacky workaround that I haven't completely thought through but which might be possible... if there is a way to manually provide the certificate details to your curl command (i.e. in your code) or tell the curl command to ignore the certificate validation (curl -k or curl --insecure on the command line), then that could also allow your code to work. Definitely not the preferred approach... I would recommend talking to your server admin, as this is a problem you will encounter anytime you want to use the Connect Server API in the future.

#15

Hi,

Below is our initial attempt to collect the logs and also the Shiny usage info with a user appended. It would be helpful to have the app names available too. I think the logs have the app id and the shiny stuff has the content_guid. Any chance of getting decodes for both added to the API in future releases?

if(!success) {install.packages("pacman")
  require(pacman)
}

pacman::p_load(jsonlite, httr, curl, dplyr, data.table)

connectServer <- ""
apiKey <- ""
# set up environment
# Note that the connectServer string must have a trailing slash

# get audit logs
authHeader <- add_headers(Authorization = paste("Key", apiKey))
resp <- GET(
  paste0(connectServer, "__api__/v1/audit_logs?ascOrder=true&limit=500"),
  authHeader
)
payload <- content(resp)
# print first 25!
# print(payload$result)
# now step through the remaining audit logs

log <- rbindlist(payload$result)

while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  # print(payload$result)
  
  log <- rbind(log, rbindlist(payload$result))
 
}

# Get Shiny App Usage
authHeader <- add_headers(Authorization = paste("Key", apiKey))
resp <- GET("localhost:3939/__api__/v1/instrumentation/shiny/usage?ascOrder=true&limit=500",
            authHeader)

payload <- content(resp)

shiny_usage <- rbindlist(payload$result)

while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  # print(payload$result)
  shiny_usage <- rbind(shiny_usage, rbindlist(payload$result))
}

# Get Users
users <- GET("localhost:3939/__api__/v1/users?page_size=500", 
             add_headers(Authorization = paste("Key", apiKey)))

user_info <- fromJSON(content(users, "text"))

user_info <- data.frame(user_info$results)

user_info

# Join Shiny Usage and Users
shiny_usage <- left_join(shiny_usage, user_info[c("username","guid")], by = c("user_guid"="guid"))
#16

Thanks Cole and sorry for my late response; It does throw me SSL error as I try to do curl from the RSonnect server.

I am using GET(url, add_headers(Authorization = apikey))) to connect from my desktop RStudio; its working fine. Do we have any option to certificate over here ?

Regards,
Ankur

#17

My recommendation would definitely be to not use a self-signed certificate. A certificate generated by your company's Certificate Authority (ask your IT department, as most companies have one) will likely be automatically trusted by Connect itself (as well as all other computers on your network), and so is a much better user experience.

If you have no choice but to use the self-signed certificate, your options are to:

  • somehow pass the certificate along with your app so that you can manually declare trust (not great)
  • find a way to tell the curl package not to worry about validating certificates (not great)
  • import the self-signed certificate into your Connect server's set of trusted certificates (This is an example SO post on the topic of RedHat )
#18

Very nice! Thanks for sharing!!

A couple of things:

There is definitely some exciting stuff out there to be explored!

EDIT: Oh, there are also some bash examples of using these deployment / content APIs here:

#19

Thank you, I will go through the release docs :slight_smile:

Also, I have fixed my code, added config option and its working good in Rstudio Connect server.

#20

Thank you :slight_smile:

How would you map App_guid with App_ID?