Development version of readr failing to compile

readr
tidyverse
package-installation
#1

Is this an out of memory problem, or something else?

(If it is memory, am I out of luck? Doesn't seem like caching an arbitrary dev version would be desirable...)

> devtools::install_github("tidyverse/readr")
Downloading GitHub repo tidyverse/readr@master
✔  checking for file ‘/tmp/RtmptWF1Cl/remotes31d734f1f66/tidyverse-readr-69c9fd3/DESCRIPTION’ ...
─  preparing ‘readr’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  looking to see if a ‘data/datalist’ file should be added
─  building ‘readr_1.2.0.tar.gz’
   
Installing package into ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5’
(as ‘lib’ is unspecified)
* installing *source* package ‘readr’ ...
** libs
g++  -I"/opt/R/3.5.0/lib/R/include" -DNDEBUG  -I"/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/Rcpp/include" -I"/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/BH/include" -I/usr/local/include   -fpic  -g -O2  -c Collector.cpp -o Collector.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make: *** [Collector.o] Error 4
/opt/R/3.5.0/lib/R/etc/Makeconf:168: recipe for target 'Collector.o' failed
ERROR: compilation failed for package ‘readr’
* removing ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/readr’
* restoring previous ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/readr’
Error in i.p(...) : 
  (converted from warning) installation of package ‘/tmp/RtmptWF1Cl/file31d61b9028d/readr_1.2.0.tar.gz’ had non-zero exit status
Development version of dplyr failing to compile
#2

Yes. It looks like the compile process is running out of memory:

g++: internal compiler error: Killed (program cc1plus)
#3

@andy_kipp looks like this also is taking out the tidyverse install from binary path on RStudio Cloud.

Any plans on addressing this issue?

c.f.

> install.packages("tidyverse")
Installing package into ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5’
(as ‘lib’ is unspecified)
also installing the dependencies ‘haven’, ‘readr’

trying URL 'http://cran.rstudio.org/src/contrib/haven_2.0.0.tar.gz'
Content type 'application/x-gzip' length 204563 bytes (199 KB)
==================================================
downloaded 199 KB

trying URL 'http://cran.rstudio.org/src/contrib/readr_1.2.1.tar.gz'
Content type 'application/x-gzip' length 358690 bytes (350 KB)
==================================================
downloaded 350 KB

trying URL 'http://package-proxy/src/contrib/tidyverse_1.2.1.tar.gz'
Content type 'application/x-tar' length 87568 bytes (85 KB)
==================================================
downloaded 85 KB

* installing *source* package ‘readr’ ...
** package ‘readr’ successfully unpacked and MD5 sums checked
** libs
g++  -I"/opt/R/3.5.0/lib/R/include" -DNDEBUG  -I"/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/Rcpp/include" -I"/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/BH/include" -I/usr/local/include  -I. -Ircon -fpic  -g -O2  -c Collector.cpp -o Collector.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make: *** [Collector.o] Error 4
/opt/R/3.5.0/lib/R/etc/Makeconf:168: recipe for target 'Collector.o' failed
ERROR: compilation failed for package ‘readr’
* removing ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/readr’
Warning in install.packages :
  installation of package ‘readr’ had non-zero exit status
ERROR: dependency ‘readr’ is not available for package ‘haven’
* removing ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/haven’
Warning in install.packages :
  installation of package ‘haven’ had non-zero exit status
* installing *binary* package ‘tidyverse’ ...
* DONE (tidyverse)

The downloaded source packages are in
	‘/tmp/RtmpgYI6b1/downloaded_packages’
#4

So as far as I can tell, readr 1.2 hit CRAN just at the start of a major holiday weekend in the US. In the past, packages that can't compile within Cloud's (alpha-version) memory limit have been built and cached. Although I have no inside knowledge, I expect that as soon as anybody is available to do so, this will happen for the latest readr as well. But in the meantime, I think anything that depends on readr is unfortunately going to fail to install in Cloud. This has already come up once this weekend:

As I did in that case, I've set up a workaround if you need to install tidyverse or any other readr-dependent package before somebody manages to build and cache readr 1.2.1: a Cloud project with up-to-date tidyverse except for readr (readr is version 1.1.1). You should be able to copy this project into your Cloud and go from there:

https://rstudio.cloud/project/144630

FWIW, since this latest issue is about CRAN readr, I see it as somewhat separate from my question above — right now, I don't really expect the Cloud team to cache binaries of arbitrary development versions of packages. Seems like that would get out of hand quickly! I am curious whether there is a longer term plan for how to handle all this in Cloud's future, but the mere existence of my curiosity doesn't oblige anybody else to come up with an answer! :grin:

#5

Hi! readr 1.2.1 has now been cached! and yes, we do have a plan on our roadmap for how to deal with the caching of new versions of cran packages as they are released!

#6

actually, scratch my last (about readr 1.2.1 being cached). I found a bit of an inconsistency with how these packages are sourced -- sorry about that!

#7

readr 1.2.1 is now cached properly!

Issue with installing the "drc" package
#8

2 posts were split to a new topic: After installing readr 1.2.1, unable to load shared object readr.so

#9

I'm facing the same issue today, is this because of 1.3.1?

#10

1.3.1 should now be cached as well.

#11

Spoke too soon, investigating why it did not cache.

#12

I'm facing the same memory issue when trying to install tidyverse dependencies

g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions. make: *** [Collector.o] Error 4 /opt/R/3.5.0/lib/R/etc/Makeconf:168: recipe for target 'Collector.o' failed ERROR: compilation failed for package ‘readr’ * removing ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.5/readr’ Warning in install.packages : installation of package ‘readr’ had non-zero exit status

Best wishes,

Paulo

#13

readr 1.3.1 should now actually be cached.