keras not found even after installing

[dnaeye]

I can't figure out how why the keras package isn't installing correctly for me. I did:

install.packages("keras")
library(keras)
install_keras()

But when I try running model_keras <- keras_model_sequential(), I get the following error:

ImportError: No module named keras
Use the install_keras() function to install the core Keras library
Error: Error loading Python module keras

I saw that py_config() was one way to see what could be wrong, and this is what I saw:

python:         /usr/bin/python
libpython:      /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome:     /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version:        2.7.10 (default, Aug 17 2018, 19:45:58)  [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
numpy:          /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version:  1.8.0
keras:          [NOT FOUND]

python versions found: 
 /Users/dhong/anaconda/envs/r-tensorflow/bin/python
 /usr/bin/python
 /usr/local/bin/python3
 /Users/dhong/anaconda/bin/python
 /Users/dhong/anaconda3/bin/python

The references to Python 2.7 are strange because when I run python -V in Terminal (I'm on Mac), I get the following: Python 3.6.4 :: Anaconda, Inc.

Someone please help a newb out! This is driving me crazy!

[dnaeye]

I think I accidentally typed those commands out of order when creating this post, so I've corrected this typo.

In any case, I re-ran all three commands, but it still didn't work.

[alistaire]

This looks like a PATH issue—it's using the system Python, which is probably not the one Keras is installed in (which I suspect is the r-tensorflow conda environment). You can set PATH in ~/.Renviron or select what Python environment to use with reticulate before loading anything.

[dnaeye]

@alistaire That's what I suspected. I tried using use_python("/Users/dhong/anaconda/envs/r-tensorflow/bin/python") and then going through the whole keras install routine, but that didn't work either. How do I set the PATH? By the way, I'm on a Mac.

[alistaire]

The general way to set PATH is in ~/.Rprofile, which gets evaluated when an R session is started up. It has to be a full string there; no bash additions, so something like

PATH=/usr/local/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin

Check what it is at any given point with Sys.getenv('PATH'). You can also use reticulate::use_condaenv and the like, but they have to be called early, and I've had issues with them before.

The solution here is probably being more explicit with install_keras—tell it where you want it to install so you know what Python environment to activate.

For debugging purposes, making system calls from the R console can be helpful, e.g. system('which python') (or which conda) or system('pip freeze').

[dnaeye]

I can't find the .Rprofile file. I ran file.path(Sys.getenv("HOME"), ".Rprofile") which returned //Users/dhong/.Rprofile but when I go there and look through all the files (visible and hidden), there's nothing there called .Rprofile. Do I need to create it?

When you say I should probably be more explicit with install_keras how do I specify the Python environment?

[alistaire]

.Rprofile != .Renviron; the former is an R script that is run at the beginning of each session (try putting cat('Hello world!') in there and restarting your R session), while the latter is just a list of environment variables. They don't exist by default, so you'll have to create them in your home directory (thus the ~/).

You can do this from the command line with something like nano ~/.Renviron, or from any plaintext file editor (including RStudio) by saving a file with the exact filepath and name. If you have devtools (and consequently usethis) installed, you can open them automagically with usethis::edit_r_profile() and usethis::edit_r_environ(), which is handy.

See ?install_keras for its options, but given you have conda installed already, you probably want to specify method = 'conda'. Alternatively, you can just install Keras with conda directly from the command line (in any particular conda environment you like) with conda install keras and then specify to use that Python environment before calling it from R, either via the PATH or with reticulate.

[dnaeye]

To confirm, I have to:

1. Run Sys.getenv('PATH') to copy the path.
2. Run usethis::edit_r_profile()
3. Edit the file by adding PATH=/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/opt/X11/bin
4. Restart RStudio
5. Run `install_keras(method = 'conda')

If the above is what you meant, it didn't work.

[alistaire]

edit_r_profile()

Nope, if you're writing it like that, it should go in .Renviron. .Rprofile is an R script, so you'd have to write it as

Sys.setenv(PATH = `/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/opt/X11/bin')

There's a difference in order of evaluation between the two files, but I doubt it would matter beyond precedence if you set an environment variable in both.

The key reason to keep environment variables in .Renviron is so you know where to look for them later when you need to change them. I'm pretty confident it's where Marie Kondo would say to put them.

[dnaeye]

That seems to have done the trick! (Either that or me setting a bunch of other random things.)

But now when I try to fit my keras model to the data, I get a fatal error. I can't copy the entire error message because there's a pop-up window preventing me, but the error message says something like:

I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

Any ideas on how to fix this?

[Yarnabrina]

I think this will be helpful:

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2

[alistaire]

the error message says something like:

I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

That's a warning, not an error. (It's just telling you your computer is capable of running TensorFlow faster if you put in some non-negligible effort.) If it's actually erroring out, that's not where it's happening.

[dnaeye]

RStudio crashes every time it hits this error message.

[alistaire]

That's not the actual error; think of it as a welcome warning to let you know TensorFlow is running. If that's all you can see, the error may not be coming through from Python, which would make fixing it hard.

[dnaeye]

Do you think reinstalling RStudio would fix my problems?

[alistaire]

Without knowing what's causing the issue, it's hard to say. The preview build is crashing on me a lot right now (pretty randomly), so it may be some sort of larger stability issue. Maybe an RStudio dev can chime in and help.