Activate conda environment in project on RStudio Cloud

[friesewoudloper]

I would like to use Python to manage my projects on a remote MLflow server. So I installed Miniconda, mlflow, {reticulate} and {mlflow}. Everything works fine. For instance, I can log experiments to the MLFlow Tracking server without any problems.

On my laptop I can create a Python file in RStudio and run this code without any problems:

import mlflow
from mlflow.tracking import MlflowClient
client = MlflowClient()
mlflow.set_tracking_uri("https://adb-XXXX.azuredatabricks.net")

from pprint import pprint
client = MlflowClient()
for rm in client.list_registered_models():
    print(dict(rm), indent=4)

When I try to do the same in my project on RStudio Cloud, I get an error:

ModuleNotFoundError: No module named 'mlflow'

However, I'm sure mlflow has been installed in the r-mlflow-1.17.0 conda environment. I followed exactly the same installation procedure on RStudio Cloud as on my laptop. I verified that I can log experiments from RStudio Cloud to the remote tracking server. So MLflow is definitely available in the conda environment.

I think the wrong conda environment is being activated, but I don't know how to change that. I tried

use_miniconda("/home/rstudio-user/.local/share/r-miniconda/envs/r-mlflow-1.17.0/bin/python", required = TRUE)

but this generates an error:

Error in use_condaenv(condaenv = condaenv, conda = miniconda_conda(),  : 
  Unable to locate conda environment '/home/rstudio-user/.local/share/r-miniconda/envs/r-mlflow-1.17.0/bin/python3.8'. 

The path to the Python executable is not correct. The version number 3.8 should be dropped, I think.

If somebody could point me to a solution I would really appreciate it.

==== Some additional information ====

reticulate::py_config()

python:         /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python
libpython:      /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/libpython3.6m.so
pythonhome:     /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate:/home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate
version:        3.6.13 | packaged by conda-forge | (default, Feb 19 2021, 05:36:01)  [GCC 9.3.0]
numpy:          /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/numpy
numpy_version:  1.19.5

reticulate::conda_list()

                                             name                                                                                                      python
1 mlflow-da39a3ee5e6b4b0d3255bfef95601890afd80709 /home/rstudio-user/.local/share/r-miniconda/envs/mlflow-da39a3ee5e6b4b0d3255bfef95601890afd80709/bin/python
2                                 r-mlflow-1.17.0                                 /home/rstudio-user/.local/share/r-miniconda/envs/r-mlflow-1.17.0/bin/python
3                                    r-reticulate                                    /home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python

Executing

reticulate::use_miniconda("r-mlflow-1.17.0", required=T)

results in:

ERROR: The requested version of Python ('/home/rstudio-user/.local/share/r-miniconda/envs/r-mlflow-1.17.0/bin/python') cannot be used, as another version of Python
('/home/rstudio-user/.local/share/r-miniconda/envs/r-reticulate/bin/python') has already been initialized. Please restart the R session if you need to attach reticulate to a
different version of Python.
Error in use_python(python, required = required) : 
  failed to initialize requested version of Python

I get this error even if I first restart my R session.

[samp]

Hello

If you can you provide your project id, I'll see if i can reproduce this problem.

thanks
sam