Can't install xgboost from CRAN

#1

Hi, this could be an xgboost error and not be related to Rstudio.
I tried to install xgboost and got the error below.
Is this a known issue on rstudiocloud?

Thank you
Richard

install.packages("xgboost")
Installing package into ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.4’
(as ‘lib’ is unspecified)
trying URL 'http://package-proxy/src/contrib/xgboost_0.6.4.1.tar.gz'
Content type 'application/x-gzip' length 596978 bytes (582 KB)
==================================================
downloaded 582 KB

* installing *source* package ‘xgboost’ ...
** package ‘xgboost’ successfully unpacked and MD5 sums checked
configure: creating ./config.status
config.status: creating src/Makevars
** libs
g++ -std=gnu++11 -I/opt/R/3.4.3/lib/R/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_  -I/usr/local/include  -fopenmp  -fpic  -g -O2 -c xgboost_R.cc -o xgboost_R.o
g++ -std=gnu++11 -I/opt/R/3.4.3/lib/R/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_  -I/usr/local/include  -fopenmp  -fpic  -g -O2 -c xgboost_custom.cc -o xgboost_custom.o
gcc -I/opt/R/3.4.3/lib/R/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_  -I/usr/local/include   -fpic  -g -O2  -c xgboost_assert.c -o xgboost_assert.o
gcc -I/opt/R/3.4.3/lib/R/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_  -I/usr/local/include   -fpic  -g -O2  -c init.c -o init.o
g++ -std=gnu++11 -I/opt/R/3.4.3/lib/R/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_  -I/usr/local/include  -fopenmp  -fpic  -g -O2 -c amalgamation/xgboost-all0.cc -o amalgamation/xgboost-all0.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make: *** [amalgamation/xgboost-all0.o] Error 4
/opt/R/3.4.3/lib/R/etc/Makeconf:166: recipe for target 'amalgamation/xgboost-all0.o' failed
ERROR: compilation failed for package ‘xgboost’
* removing ‘/home/rstudio-user/R/x86_64-pc-linux-gnu-library/3.4/xgboost’
Warning in install.packages :
  installation of package ‘xgboost’ had non-zero exit status

The downloaded source packages are in
	‘/tmp/RtmpqW44O4/downloaded_packages’
#2

When you get an error, compiler or otherwise, one of the things to do is to do a search for some fragments of the error message you received.

For example searching for "recipe for target 'amalgamation/xgboost-all0.o' failed" finds an Stack Overflow discussing the error you appear to have run into.

#3

Thank you. I will try that. I just thought that maybe something could be improved in the general set-up of the rstudio cloud service concerning xgboost. I didn't try and risk my current local installation but I think I did not get the error on my local machine on Windows..

#4

That output often implies that g++ ran out of memory while attempting to compile the associated source file. There are typically two ways around this:

  1. Add more memory to the machine,
  2. Modify compilation flags to require less memory (e.g. don't compile translation units in parallel). This is typically done by unsetting the MAKEFLAGS environment variable.

How much memory is available for the R session attempting installation of xgboost?

#5

@kevinushey How can I check this on rstudio.cloud? I only encounter this problem in the cloud. Is this the right spot to address such issues? I would really love to use rstudio.cloud as my online plattform for demanding machine learning tasks (in the future, I am aware that it is in alpha state). So far it crashes when I try to run my production scripts. Standard packages work fine. Now I try to install more special ones. Thank you!

#6

Similar when I tried

library(keras)
install_keras()

after I have installed the package. I got the following:

Creating virtualenv for TensorFlow at  ~/.virtualenvs/r-tensorflow 
New python executable in /home/rstudio-user/.virtualenvs/r-tensorflow/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python
Upgrading pip ...
Collecting pip
  Using cached pip-9.0.2-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-9.0.2
Upgrading wheel ...
Collecting wheel
  Using cached wheel-0.30.0-py2.py3-none-any.whl
Installing collected packages: wheel
Successfully installed wheel-0.30.0
Upgrading setuptools ...
Collecting setuptools
  Using cached setuptools-39.0.1-py2.py3-none-any.whl
Installing collected packages: setuptools
Successfully installed setuptools-39.0.1
Installing TensorFlow ...
Collecting tensorflow
  Downloading tensorflow-1.6.0-cp27-cp27mu-manylinux1_x86_64.whl (45.9MB)
/bin/bash: line 1:  1259 Killed                  '/home/rstudio-user/.virtualenvs/r-tensorflow/bin/pip' install --ignore-installed --upgrade 'tensorflow' 'h5py' 'pyyaml' 'requests' 'Pillow' 'keras' 'scipy'
Error: Error 137 occurred installing TensorFlow

I guess I would have to configure rstudio.cloud but I don't know how. I would be thankful for some reference. Thank you and regards!
PS: I could open up a new "package topic" and not add it to the other question if you prefer.

#7

Does it make any difference if you attempt compilation after restarting the R session, and clearing the workspace of any large R objects?

What is the output of

cat(readLines("/proc/meminfo"), sep = "\n")

on your instance?

#8

@kevinushey after deleting all objects and a restart of the session I was able to install xgboost! Thank you!

#9

@kevinushey and keras as well!