CRAN Mirror, RStudio, and Versioning

[wburr]

I'm mainly throwing this here to hopefully flag it to the attention of someone who works on the CRAN mirror, or is responsible for setting things up on the RStudio defaults.

I'm working with 650 first-year undergraduates getting RStudio installed on their personal laptop, so I get to see every way that something can fail. And this morning, something has failed in a way that should be avoidable:

1. Try to install rmarkdown (install.packages)
2. Directs to the CRAN mirror
3. Insists on updating stringi
4. Can't find stringi 1.5.3 ... 404 error

I check the cran.rstudio.com/bin/macosx/contrib/4.0/ ... and it's 404ing because it's NOT THERE.

So basically, having a package insist on an update to another package, and then directing to your mirror, but the mirror not actually having that version is really annoying. Especially for a first-year student who has no idea what this might be. And now I'm debugging dozens of Mac users, getting them to workaround the issue and explaining mirrors and repos to them.

Not sure what the solution is, if anything - I know mirroring takes time. But if there's going to be a cloud repo redirect managed by you folks, you can't let it ever not be fully up to date. A default doesn't work if it ... doesn't work.

Edit As of 12:42 stringi was compiled by CRAN and is of course available as a binary again. I think overall, though, this points to some very weird issue in the installation path - why does installing RMarkdown and then knitting force the installation of the latest stringi? As far as I can tell, the dependencies on rmarkdown 2.3 isn't stringi 1.5.6, so what happens to trigger that? And can it be avoided so we don't get windows where binaries aren't available. There's a lot of novice users for whom installation of Command Line Tools on Mac (or RTools on Windows) is a step too far.

[wburr]

And here's what forces this whole issue:

[rstub]

If you look at the official CRAN page for that package at https://cran.r-project.org/package=stringi, you will see that CRAN has not build the latest version of this package for current R on Mac OS:

In principle R should handle this transparently and fall back to compiling the package from source. That can be complex for stringi, though.

BTW, I am not sure why an update should be necessary: rmarkdown depends on stringr, which depends on stringi ≥ 1.1.7. Not sure if another versioned dependency comes in via a different route. So it might be sufficient to install stringi version 1.4.6 which is available as binary now.

[wburr]

In theory this should work, but unfortunately it doesn't. It's only popped up in the last 12 hours - I've been getting students installed all week, and it only started this morning, and only on Mac OSX. You're right that if rmarkdown is already installed, it works fine, but something is triggering the requirement to update stringi that wasn't caught in the initial install.packages().

For now, I'm having students go directly to CRAN, download the .tar.gz, and install from package as source, but this has the extra step of requiring Mac users to install the Command Line Tools, which for first-years, is tedious and scary.

[nirgrahamuk]

I'm thinking this might be a usecase for renv, to catalog the required package version for a project.
If these are for sure available for Mac (i.e. the individual that wrote the renv.lock is a mac user), then other mac users should be able to rely on renv to reproduce the environment of the author

[rstub]

Does it help if they use

devtools::install_version("stringi", version="1.4.6")

before installing rmarkdown? I still see no reason why the most recent version of stringi should be required.

Using renv as suggested by @nirgrahamuk is also a good, even independent from these particular problems.

[wburr]

RMarkdown is installed. This is when they try to hit Knit - it does some kind of check at that point, and requires an update. I can't identify exactly what causes it to trigger - first time knitting? I setup a fresh Mac OSX copy to record a video for the students last week, and it installed 1.4.6 ok, and then knit fine both then, and still does. But any student trying from scratch runs into this as of this morning. I don't know what's requiring the latest, greatest stringi?

[wburr]

To the devtools suggestion: that introduces more package installation steps. I was really going for a minimal set of steps to get students to the point that they can create and knit to PDF R Markdown documents. These are neophytes. Anything involving the console is already a stretch. You're right that force-installing the 1.4.6 version should install.

However, I think I'm still right in saying this issue shouldn't happen - why was 1.5.3 or whatever even being requested? Just grab the latest non-source version, whatever it is. Why request the upgrade unless a dependency lists it?

[rstub]

It is difficult to answer these questions without knowing more about the host computers, e.g. which versions of the packages had been installed before and which R distribution is in use. Some general notes:

• The RStudio IDE requires some some packages for certain functions with certain mimimum package versions. For example, rmarkdown is needed for processing .Rmd files. These requirements are checked when the appropriate function is used. The full list of package requirements is in the documentation: https://docs.rstudio.com/ide/server-pro/r-package-dependencies.html

• Whether binary or source packages are used is independent of the RStudio IDE and is only controlled by R itself. See the "Binary Packages" section in the documentation for install.packages() for details.

[rstub]

One further note: It seems there has been a particular inconsistency on CRAN w.r.t. stringi, c.f. https://stat.ethz.ch/pipermail/r-devel/2020-September/079952.html

[wburr]

That does match my experience: it fixed itself in about six hours, once the binary finished compiling. I only noticed/cared because I was in the middle of helping hundreds of people run through the initial RStudio+R -> packages -> rmarkdown -> knit sequence, and a large number use Macs, and hit that issue because we chose to not set them up to compile-from-source.

So not an issue with RStudio, just an issue with the binary/not binary package ecosystem. I saw a similar issue with backports last year, where there was a 12-24 hour window where it was broken because of an update on one thing but not all of the dependencies.

Feel free to close/lock the thread, @rstub.