Following the packrat "How to Set Up a Custom CRAN-like Repository" and not recieving expected output for `write_PACKAGES`

I'm following through the Packrat CRAN link tutorial listed here. I believe that until:

tools::write_PACKAGES(contribDir, type = "source")

Everything is functioning correctly. I have a package from pkgKitten called sashimi in a temp folder, and have copied it's tar.gz to the path in the contribDir variable. The call to tools::write_PACKAGES() has completed and populated contribDir with ‘PACKAGES’, ‘PACKAGES.gz’ and ‘PACKAGES.rds’ files.

The next command is to:

lapply(binPaths, function(path) {
  tools::write_PACKAGES(path)
})

I believe this iterates over binPaths, checks for packages in those directories and then indexes them for each iteration, to create the 3 files mentioned above.

However, I don't see where those folders get populated with the sashimi material in the tutorial. contribDir puts the tar.gz into the */scr/contrib folder, but the */bin/*/... folder remain empty, so when passed to write_PACKAGES, there is no creation of the index files, and so the final install step doesn't work.

Have I missed something?

Hi @DaveRGP, what's is the value of binPaths at the point that you run the lapply routine?

1 Like
$win.binary
[1] "C:/Users/dparr/Documents/local-cran/bin/windows/contrib/3.4"

$mac.binary
[1] "C:/Users/dparr/Documents/local-cran/bin/macosx/contrib/3.4"

$mac.binary.mavericks
[1] "C:/Users/dparr/Documents/local-cran/bin/macosx/mavericks/contrib/3.4"

$mac.binary.leopard
[1] "C:/Users/dparr/Documents/local-cran/bin/macosx/leopard/contrib/3.4"

The return looks like:

[1] 0

$mac.binary
[1] 0

$mac.binary.mavericks
[1] 0

$mac.binary.leopard
[1] 0

Which I'm assuming means "There are no packages so I'm giving 0 output"

Ok, I was just able to test this and got it to work by replacing this command:

lapply(binPaths, function(path) {
  tools::write_PACKAGES(path)
})

With this one:

lapply(binPaths, function(path) {
  file.copy(file.path(contribDir, "PACKAGES"),
            path)
  file.copy(file.path(contribDir, "PACKAGES.gz"),
            path)
})

@kevinushey - Can you take a look at this and let me know if I'm breaking something? :slight_smile:

1 Like

It looks like the behavior of tools::write_PACKAGES() must have changed with R 3.4.2, since these same steps work with R 3.3.x.

In any case, all you need is an empty PACKAGES file in those directories. You do not want to just copy the same PACKAGES file from the source repository, since your binary repositories are not actually serving any packages (at least, not at this point).

You might be able to ignore the stuff related to binary repositories altogether if you just enforce options(pkgType = "source") within your R sessions, though. But I vaguely recall some R APIs expect that binary repositories exist and have this basic structure, even when opting to use source packages for installation.

1 Like

Thanks both, that's pretty useful.

If as @kevinushey says, there may be an expectation for the binaries to be present, is there a way within the package building process to build the correct binaries and deploy them into the structure? Can I use something like devtools::build(binary = TRUE) in the sashimi directory, and move the output into the correct folders, then run the original lapply?

I think in general you need a Windows machine to produce Windows binaries for R packages, and a macOS machine to produce macOS binaries. (You might be able to get away with ignoring that for R packages that don't contain any compiled C / C++ code, but I am not sure)

1 Like

So right now the tutorial errors out in the last step with:

install.packages("sashimi", type = "source")
Installing package into ‘E:/Dave/Analysis/packratDemo/packrat/lib/x86_64-w64-mingw32/3.4.1’
(as ‘lib’ is unspecified)
Warning in install.packages :
 cannot open compressed file '//C:/Users/dparr/Documents/local-cran/src/contrib/PACKAGES', probable reason 'Invalid argument'
Error in install.packages : scheme not supported in URL '/no/such/path/here/i/hope//PACKAGES'

I'm not sure why the "no such path here I hope" thing is returned by the Error, that's not my joke :slight_smile:

My actual repos list is:

getOption("repos")
                                               BioCsoft                                                 BioCann 
           "https://bioconductor.org/packages/3.4/bioc" "https://bioconductor.org/packages/3.4/data/annotation" 
                                                BioCexp                                               BioCextra 
"https://bioconductor.org/packages/3.4/data/experiment"           "https://bioconductor.org/packages/3.4/extra" 
                                                   CRAN                                               CRANextra 
                            "https://cran.rstudio.com/"                   "https://www.stats.ox.ac.uk/pub/RWin" 
                                                  sushi 
           "file://C:/Users/dparr/Documents/local-cran" 

Copying the three files named in the documented tutorials still gives the same error, as does deleting those contents to make the files 'empty'.

Oddly, I initially thought that the issue was because the write_PAKAGES() wasn't filling out the bin folders while the tutorial said it would. However I've just noticed the error is actually from reading the src folder, which should be correctly populated. Any other suggestions?

Update:
I just did this on a macbook running Sierra 10.12.4 (16E195), and it worked. The issue is on a windows machine running Windows Server 2012 R2.

The error seems to be in the creation of the cranUI variable. Removing the file:\\ prefix changes the output to:

Installing package into ‘E:/Dave/Analysis/packratDemo/packrat/lib/x86_64-w64-mingw32/3.4.1’
(as ‘lib’ is unspecified)
Warning in install.packages :
  unable to access index for repository C:/Users/dparr/Documents/local-cran/src/contrib:
  scheme not supported in URL 'C:/Users/dparr/Documents/local-cran/src/contrib/PACKAGES'
Warning in install.packages :
  package ‘sashimi’ is not available (for R version 3.4.1)

Do you have any better luck with the development version of Packrat?

devtools::install_github("rstudio/packrat")

Same return as above :confused: