keras not found even after installing

dnaeye · January 21, 2019, 5:16am

I can't figure out how why the keras package isn't installing correctly for me. I did:

install.packages("keras")
library(keras)
install_keras()

But when I try running model_keras <- keras_model_sequential(), I get the following error:

ImportError: No module named keras
Use the install_keras() function to install the core Keras library
Error: Error loading Python module keras

I saw that py_config() was one way to see what could be wrong, and this is what I saw:

python:         /usr/bin/python
libpython:      /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome:     /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version:        2.7.10 (default, Aug 17 2018, 19:45:58)  [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
numpy:          /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version:  1.8.0
keras:          [NOT FOUND]

python versions found: 
 /Users/dhong/anaconda/envs/r-tensorflow/bin/python
 /usr/bin/python
 /usr/local/bin/python3
 /Users/dhong/anaconda/bin/python
 /Users/dhong/anaconda3/bin/python

The references to Python 2.7 are strange because when I run python -V in Terminal (I'm on Mac), I get the following: Python 3.6.4 :: Anaconda, Inc.

Someone please help a newb out! This is driving me crazy!

Yarnabrina · January 21, 2019, 4:12am

I think you should try this:

install.packages("keras")
library(keras)
install_keras()

You switched the 2nd and 3rd line.

You can refer to this page:

Hope this helps.

dnaeye · January 21, 2019, 5:18am

I think I accidentally typed those commands out of order when creating this post, so I've corrected this typo.

In any case, I re-ran all three commands, but it still didn't work.

alistaire · January 21, 2019, 6:00am

This looks like a PATH issue—it's using the system Python, which is probably not the one Keras is installed in (which I suspect is the r-tensorflow conda environment). You can set PATH in ~/.Renviron or select what Python environment to use with reticulate before loading anything.

dnaeye · January 21, 2019, 2:52pm

@alistaire That's what I suspected. I tried using use_python("/Users/dhong/anaconda/envs/r-tensorflow/bin/python") and then going through the whole keras install routine, but that didn't work either. How do I set the PATH? By the way, I'm on a Mac.

alistaire · January 21, 2019, 6:29pm

The general way to set PATH is in ~/.Rprofile, which gets evaluated when an R session is started up. It has to be a full string there; no bash additions, so something like

PATH=/usr/local/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin

Check what it is at any given point with Sys.getenv('PATH'). You can also use reticulate::use_condaenv and the like, but they have to be called early, and I've had issues with them before.

The solution here is probably being more explicit with install_keras—tell it where you want it to install so you know what Python environment to activate.

For debugging purposes, making system calls from the R console can be helpful, e.g. system('which python') (or which conda) or system('pip freeze').

dnaeye · January 21, 2019, 10:57pm

I can't find the .Rprofile file. I ran file.path(Sys.getenv("HOME"), ".Rprofile") which returned //Users/dhong/.Rprofile but when I go there and look through all the files (visible and hidden), there's nothing there called .Rprofile. Do I need to create it?

When you say I should probably be more explicit with install_keras how do I specify the Python environment?

alistaire · January 21, 2019, 11:29pm

.Rprofile != .Renviron; the former is an R script that is run at the beginning of each session (try putting cat('Hello world!') in there and restarting your R session), while the latter is just a list of environment variables. They don't exist by default, so you'll have to create them in your home directory (thus the ~/).

You can do this from the command line with something like nano ~/.Renviron, or from any plaintext file editor (including RStudio) by saving a file with the exact filepath and name. If you have devtools (and consequently usethis) installed, you can open them automagically with usethis::edit_r_profile() and usethis::edit_r_environ(), which is handy.

See ?install_keras for its options, but given you have conda installed already, you probably want to specify method = 'conda'. Alternatively, you can just install Keras with conda directly from the command line (in any particular conda environment you like) with conda install keras and then specify to use that Python environment before calling it from R, either via the PATH or with reticulate.

dnaeye · January 22, 2019, 3:38pm

To confirm, I have to:

Run Sys.getenv('PATH') to copy the path.
Run usethis::edit_r_profile()
Edit the file by adding PATH=/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/opt/X11/bin
Restart RStudio
Run `install_keras(method = 'conda')

If the above is what you meant, it didn't work.

alistaire · January 23, 2019, 1:10am

Nope, if you're writing it like that, it should go in .Renviron. .Rprofile is an R script, so you'd have to write it as

Sys.setenv(PATH = `/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/opt/X11/bin')

There's a difference in order of evaluation between the two files, but I doubt it would matter beyond precedence if you set an environment variable in both.

The key reason to keep environment variables in .Renviron is so you know where to look for them later when you need to change them. I'm pretty confident it's where Marie Kondo would say to put them.

dnaeye · January 23, 2019, 3:21pm

That seems to have done the trick! (Either that or me setting a bunch of other random things.)

But now when I try to fit my keras model to the data, I get a fatal error. I can't copy the entire error message because there's a pop-up window preventing me, but the error message says something like:

I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

Any ideas on how to fix this?

Yarnabrina · January 23, 2019, 3:34pm

I think this will be helpful:

alistaire · January 24, 2019, 1:18am

That's a warning, not an error. (It's just telling you your computer is capable of running TensorFlow faster if you put in some non-negligible effort.) If it's actually erroring out, that's not where it's happening.

dnaeye · January 24, 2019, 1:20am

RStudio crashes every time it hits this error message.

alistaire · January 24, 2019, 1:23am

That's not the actual error; think of it as a welcome warning to let you know TensorFlow is running. If that's all you can see, the error may not be coming through from Python, which would make fixing it hard.

dnaeye · January 24, 2019, 1:26am

Do you think reinstalling RStudio would fix my problems?

alistaire · January 24, 2019, 1:31am

Without knowing what's causing the issue, it's hard to say. The preview build is crashing on me a lot right now (pretty randomly), so it may be some sort of larger stability issue. Maybe an RStudio dev can chime in and help.

system · February 14, 2019, 1:31am

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.