numpy.core.multiarray error : numpy version conflict in console ? (but not in terminal)

Hello,

I have a blocking issue while trying to setup a python environmnet in order to use a specific module named asammdf.

Please find below a reproducible example. If I basically setup a new virtualenv in a project and try to load the module, I can't import numpy neither asammdf (numpy.core.multiarray error).

reticulate::virtualenv_install("my_env/", packages = "asammdf==7.0.2", method = "virtualenv", ignore_installed = TRUE)
#> Using Python: /data/rstudio/projects/username/.pyenv/versions/3.8.10/bin/python3.8
#> Creating virtual environment 'my_env/' ... Done!
#> Installing packages: 'pip', 'wheel', 'setuptools', 'numpy'
#> Virtual environment 'my_env/' successfully created.
#> Using virtual environment 'my_env/' ...

# restart session here
reticulate::import("numpy", convert = FALSE)
#> Error in py_module_import(module, convert = convert): RuntimeError: The current Numpy installation ('/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py') fails to pass simple sanity checks. This can be caused for example by incorrect BLAS library being linked in, or by mixing package managers (pip, conda, apt, ...). Search closed numpy issues for similar problems.
#> 
#> Detailed traceback:
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py", line 349, in <module>
#>     _sanity_check()
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py", line 347, in _sanity_check
#>     raise RuntimeError(msg.format(__file__)) from None
reticulate::import("asammdf", convert = FALSE)
#> Error in py_module_import(module, convert = convert): ImportError: numpy.core.multiarray failed to import
#> 
#> Detailed traceback:
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/__init__.py", line 15, in <module>
#>     from .blocks.source_utils import Source
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/blocks/source_utils.py", line 10, in <module>
#>     from . import v2_v3_blocks as v3b
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/blocks/v2_v3_blocks.py", line 16, in <module>
#>     from numexpr import evaluate
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numexpr/__init__.py", line 24, in <module>
#>     from numexpr.interpreter import MAX_THREADS, use_vml, __BLOCK_SIZE1__
#>   File "/data/rstudio/projects/francois.bizet/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(

It seems to be related tu numpy version. If I try similarly with only numpy, I get a "The current Numpy installation fails to pass simple sanity checks." error. I don't have the problem if I force an older version of numpy 1.21

reticulate::virtualenv_install("my_env/", packages = c("numpy==1.22.3"), method = "virtualenv", ignore_installed = TRUE)
#> Using Python: /data/rstudio/projects/username/.pyenv/versions/3.8.10/bin/python3.8
#> Creating virtual environment 'my_env/' ... Done!
#> Installing packages: 'pip', 'wheel', 'setuptools', 'numpy'
#> Virtual environment 'my_env/' successfully created.
#> Using virtual environment 'my_env/' ...

# restart session here
reticulate::import("numpy", convert = FALSE)
#> Error in py_module_import(module, convert = convert): RuntimeError: The current Numpy installation ('/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py') fails to pass simple sanity checks. This can be caused for example by incorrect BLAS library being linked in, or by mixing package managers (pip, conda, apt, ...). Search closed numpy issues for similar problems.
#> 
#> Detailed traceback:
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py", line 349, in <module>
#>     _sanity_check()
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/numpy/__init__.py", line 347, in _sanity_check
#>     raise RuntimeError(msg.format(__file__)) from None

What I don't get is that if I open a python command line in the terminal, both imports of numpy and asammdf works nicely.

I tried a work around with asammdf and force an older version of numpy but while I can import numpy (if placed in 1st place only in module install), there is still an error in asammdf import. It seems like asammdf forcefully instally the latest numpy version, which doesn't work within my R environment (BLAS library ?).

reticulate::virtualenv_install("my_env/", packages = c("numpy==1.21.0", "asammdf==7.0.2"), method = "virtualenv", ignore_installed = TRUE)
#> Using Python: /data/rstudio/projects/username/.pyenv/versions/3.8.10/bin/python3.8
#> Creating virtual environment 'my_env/' ... Done!
#> Installing packages: 'pip', 'wheel', 'setuptools', 'numpy'
#> Virtual environment 'my_env/' successfully created.
#> Using virtual environment 'my_env/' ...

# restart session here
reticulate::import("numpy", convert = FALSE)
#> Module(numpy)
reticulate::import("asammdf", convert = FALSE)
#> Error in py_module_import(module, convert = convert): ImportError: numpy.core.multiarray failed to import
#> 
#> Detailed traceback:
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/__init__.py", line 17, in <module>
#>     from .mdf import MDF, SUPPORTED_VERSIONS
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/mdf.py", line 42, in <module>
#>     from .blocks.mdf_v2 import MDF2
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/blocks/mdf_v2.py", line 9, in <module>
#>     from .mdf_v3 import MDF3
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#>   File "/data/rstudio/projects/username/projects/2022/2022_04_python_test/my_env/lib/python3.8/site-packages/asammdf/blocks/mdf_v3.py", line 49, in <module>
#>     from .cutils import get_channel_raw_bytes
#>   File "/data/rstudio/projects/username/R/x86_64-pc-linux-gnu-library/4.0/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(

Thanks a lor for your inputs for a work around.

François

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.