How do you test your internal packages (Travis, Jenkins, by hand?)

Posit Professional & Hosted Products Package Management
1

As an R user developing an internal package, what tools do you use for automated testing. Do you have access to continuous integration software like Jenkins and Travis? Would it be helpful if RStudio Package Manager ran automated tests? Do you use something else?

2 Likes
2

Our company CI software is Jenkins. It is planned to manage to use it with R development. But currently, it is not easy.
Do you have good ressources about case ?

Also, it would be nice if the product also provide integrated testing support. However, in big company as ours, there are already infrastructure for development purpose. We call it Development factory. It contains SonarQube, NEXUS, Jenkins and Gitlab.
Everything in the company must use this to go to production, it is kind of the standard.

It is why this is not easy to defend a full integrated product specific for R, when all other langage use the Factory.

It would then be better to work on integration and plugins to work with other product than completely include everything in the product itself. Even if it is awesome feature :wink:

What do you think ?

2 Likes
3

We were using Bamboo for a while, then BitBucket Pipelines. Now we're migrating to GitLab. We've had to create custom build / test scripts for each of those products that trigger on a pushed commit to an R package repository, run tests, build packages, and (if they passed) publish a dev or release build to our CRAN-alike repository.

I agree that ideally we'd want to continue to keep our git repos for R packages in the same place as our other code. Setting up the CRAN-alike repository was the easy part compared to getting the build / deploy process working.

1 Like
4

@ajar This feedback is very helpful! Right now our plan is to allow RStudio Package Manager admins to point at certain Git repos, and then have RStudio Package Manager do all the work to watch for changes, run tests and builds, and push the artifacts into the CRAN-alike repositories it manages.

@cderv we are leaning in this direction as opposed to building plugins because:
a) there are many different potential plugins and integrations
b) some R users do not have access to build infrastructure

So out of the box we'd like to provide an integrated solution, but we realize this only further continues the "fracture" between R and your other tools, so we'd eventually also like to support integrations.

4 Likes
5

That is the dilemma ! I am currently tackling it on this topic because I am trying to make R a part of our company technology for good, as any other but all the awesome product for it are specific to R, so need more justification, and kind of the opposite message of "no worries, R is like other langage, we can put it into production".
however, considering the feature already in and those that will arrive, RSPM willl be more complete for R langage that any other tool for a specific language.

So, I understand the position and I have faith that RStudio will look after not increasing the "facture" too much. :slight_smile:

Thanks for the answer!

1 Like
6

Sean,

Any updates on the next release of RSPM and it's support for the capabilities you're mentioning? Feels like this is a piece of the puzzle that is missing to build a fully automated workflow Git Push ---> Jenkins WebHook ---> Run Tests ---> Build Package ---> Deploy to CRAN-alike repo.

...allow RStudio Package Manager admins to point at certain Git repos, and then have RStudio Package Manager do all the work to watch for changes, run tests and builds, and push the artifacts into the CRAN-alike repositories it manages

1 Like
7

Great question! We are planning to ship the 1.0 version very soon (before the end of October if all goes well, and even earlier if all goes according to plan!). Subscribe to the RStudio blog to see the announcement.

The new version includes an option to point RStudio Package Manager at a Git endpoint. You specify whether Package Manager should track commits or tags, and then Package Manager watches the endpoint and upon finding a new commit or tag:

  1. Pulls the code
  2. If you're tracking commits, it updates the version number so that each commit has a unique version #
  3. Runs R CMD build to generate a source tar file
  4. Makes the source tar file available in a repository, where users can install via install.packages

You can see an example of each mode here:

  1. Tracking Tags: https://demo.rstudiopm.com/client/#/repos/9/activity
  2. Tracking Commits: https://demo.rstudiopm.com/client/#/repos/8/activity

In the future we plan on adding two complementary features:

  1. Adding the ability to run R CMD install to create a binary version of the package that can be cached in addition to the source tar file
  2. Adding the ability to run R CMD check as part of the process and an option to only make the package available if the check passes.

Both of these features require a more complete understanding of the system requirements necessary for any given CRAN package to run on a specific Linux architecture (e.g. a Docker container). We are working on that database of system requirements now.

Additionally, because RStudio Package Manager is driven by a CLI, you could create your own workflow today, e.g.

  1. Push to Git
  2. Use a CI service like Jenkins or Travis to test and build the package
  3. Have the CI service copy the resulting package source file to RStudio Package Manager and execute the CLI command to add it to the repository

Let me know if you have any questions, and w'e love your feedback when you get a chance to try out either option.

You can also learn more:

http://docs.rstudio.com/rspm/1.0.0/admin/quickstarts.html#quickstart-local-git
http://docs.rstudio.com/rspm/1.0.0/admin/repositories.html#git-sources
https://resources.rstudio.com/webinars/introduction-to-the-rstudio-package-manager-sean-lopp (minute 34)

6 Likes
8

@slopp Awesome! I'll send some feedback once I go over through all the features.

Can I ask you to check with the team if RSPM 1.0.0-7 package is available for Debian/Ubuntu? The docs are pointing to a location on S3 bucket that cannot be resolved. Same for RHEL/CentOS

http://docs.rstudio.com/rspm/1.0.0/admin/getting-started.html#installation

wget https://s3.amazonaws.com/rstudio-package-manager/rstudio-pm_1.0.0-7.deb

--2018-10-19 12:51:22--  https://s3.amazonaws.com/rstudio-package-manager/rstudio-pm_1.0.0-7.deb
Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.98.237
Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.98.237|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-10-19 12:51:22 ERROR 404: Not Found.

Thanks!

9

Apologies. That looks to be a bad link. Try this one:

Ubuntu:
https://s3.amazonaws.com/rstudio-package-manager/ubuntu/amd64/rstudio-pm_1.0.0-7_amd64.deb

Centos:
https://s3.amazonaws.com/rstudio-package-manager/centos/x86_64/rstudio-pm-1.0.0-7.x86_64.rpm

We will get that cleaned up!

2 Likes
10

@slopp I took the new version for a test drive and so far I like what I see :slight_smile:

A bit of feedback:

  • RVersion config param: rename the config variable to something more meaningful like R_HOME? Current configuration key name is a bit misleading. See RVersion under http://docs.rstudio.com/rspm/1.0.0/admin/appendix-configuration.html

  • LDAP/AD support is high on our list of features, as well as RBAC. I'm sure this is on the roadmap.

  • R CMD install and R CMD check are two features that would help us go a long way!

  • pushing packages to other sources other than RSPM: we're running JFrog Artifactory and it would be great to be able to specify the target CRAN. At this moment, most of this stuff is done via bash scripting and calling JFrog API to push packages

  • building and running RSPM inside Docker: a small issues with systemd not being the init process (PID1), will have to refactor my Dockerfile or find another way to avoid privileged container.

Keep up the good work!

Amar

11

Awesome, I am really glad you were able to try out the new version!

LDAP/AD support is high on our list of features, as well as RBAC. I'm sure this is on the roadmap.

This is on the roadmap. Unfortunately base R's install.packages has no support for providing authentication, so we would need to use a different client to allow secure installations.

R CMD install and R CMD check

Both are certainly on the way. We are working on identifying system dependencies so that RStudio Package Manager can provide information on what is needed for packages to install.

Artifactory

You can setup Artifactory to proxy RStudio Package Manager, without doing the management twice. Would this work for you?

RSPM inside Docker

If you follow these instructions you can setup and run the RSPM binary directly without the service daemons: http://docs.rstudio.com/rspm/admin/non-root-use.html

2 Likes
12

We are also using Artifactory company-wide. Our use case is two-fold:

  1. Proxy MRAN. We don't use CRAN because we need to be able to go back to prior snapshots for reproducibility. The "generic proxy repository" is one of the oldest features of Artifactory, there is not much to set up, it works with MRAN pretty much out of the box.
  2. Publish our own R packages. Starting from v.6.1 (a few months ago) Artifactory has native support for CRAN-like repositories (meaning it maintains PACKAGES file, etc). Nice UI for browsing R packages too. See Get your R Package Repository with CRAN Integration in Artifactory. We use Jenkins for the automated process to pull from git/build the packages and, when the "do the release" condition is triggered, publish them to the repository. Jenkins supports Artifactory nicely.

Regarding authentication for install.packages. If your Artifactory is password-protected and/or uses SSL with your own certificates you can do the following. Since install.packages uses download.file under the hood all you need to do is to use curl as the method option instead of the default libcurl which is much harder to configure, then put login/password to the netrc file:

## SSL
Sys.setenv(CURL_CA_BUNDLE="/path/to/cert.crt")

artifactoryURL <- "https://artifactory.company.com/artifactory"

options(repos = c(
    CRAN = paste0(artifactoryURL, "/.../"),  # proxy to MRAN
    ours = paste0(artifactoryURL, "/.../")), # our packages
    ## To use artifactory with authentication and SSL
    download.file.method = "curl"
    download.file.extra  = "--netrc-file /location/of/netrc")
1 Like
13

This!
Also, there's a long tail of small or mixed-tech shops for whom RStudio Package Manager (or any self-hosted product) is not a good fit.
Obviously, I haven't done any market research on this, but I hope I'm not alone in willing to pay for more convenience around R CI/CD, but preferably within existing ecosystems, such as: