Python virtual environment in R shinyapp.io stopped working suddenly

shiny shinyapps.io
#1

A previously working version of Python virtual environment or shinyapps.io stopped working. It seems that it is because of automatic installation of the latest prerelease version of PIP package into the virtual environment and its incompatibility. My question is that how can I enforce virtual environment to use/ install a specific pip version? Here is the code:

  reticulate::virtualenv_create(envname = "myreticulate",python = '/usr/bin/python3')
  reticulate::virtualenv_install("myreticulate",  packages = c( 'pathlib','pandas','requests'))
  reticulate::use_virtualenv("myreticulate", required = TRUE)

Here is the Shinyappio log:
Using base prefix '/usr'
New python executable in \home\shiny.virtualenvs\myreticulate\bin\python3
Also creating executable in \home\shiny.virtualenvs\myreticulate\bin\python
Installing setuptools, pkg_resources, pip, wheel...done.
Requirement already up-to-date: wheel in \home\shiny.virtualenvs\myreticulate\lib\python3.5\site-packages (0.34.2)
Requirement already up-to-date: pip in \home\shiny.virtualenvs\myreticulate\lib\python3.5\site-packages (20.1b1)
: Requirement already up-to-date: setuptools in \home\shiny.virtualenvs\myreticulate\lib\python3.5\site-packages (46.1.3)
Using virtual environment 'myreticulate' ...
Error in value3L : invalid version specification ‘20.1b1

#2

I have the same problem!! Thank you for submitting this. I have noticed the problem about one day back! My application log is exactly the same!

2 Likes
#3

I ran into the same issue. I think you are right that it's related to a very recent pip release (http://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html).

I am trying to address it by reverting back to an older version of pip in the virtualenv_install function after the most recent release is automatically loaded via the virtualenv_create function. It works (i.e., pip is successfully downgraded to older version) when I run my app locally, but when I deploy it I get the same error in the shinyapps.io log.

Here's my current approach, for reference. I will update this if I find a solution.

reticulate::virtualenv_create(envname = "python37_env", python = "python3")
reticulate::virtualenv_install("python37_env", packages = c("pulp", "statistics", "pip==20.0"), ignore_installed = TRUE)
reticulate::use_virtualenv("python37_env", required = TRUE)

Good luck, and hopefully this gets resolved soon!

2 Likes
#4

I actually found a solution for this issue. Since the bugged version of pip gets installed as soon as your create the virtualenv, I forcibly uninstalled it and then installed the version that worked when I built my app. Here is the code that I used:

virtualenv_create(envname = "python_environment", python = "python3")
virtualenv_remove(envname = "python_environment", packages = "pip")
virtualenv_install(envname = "python_environment", packages = c("pip==19.0.3","numpy","nltk","torch"), ignore_installed = TRUE)
use_virtualenv("python_environment", required = TRUE)
3 Likes
#5

Thank you so much for this solution, works for me too!

#6

Hi,
I'm experiencing the same problem. However, the solution of uninstalling and reinstalling the older version of pip creates new errors when installing other packages such as numpy and pandas - the app runs fine locally so there shouldn't be syntax errors in the code. Any ideas what could be causing this?
This is the Shiny log I get when implementing the above solution: i.e.:

virtualenv_create(envname = "python_environment", python= "python3")
virtualenv_remove(envname = "python_environment", packages = "pip")
virtualenv_install("python_environment", packages = c('pip==19.0.3', 'langdetect', 'pandas','numpy','googletrans', 'xlrd'), ignore_installed = TRUE)
use_virtualenv("python_environment", required = TRUE)```

**The Shiny log displays the following: **
**Uninstalling pip-20.2b1:**
**Successfully uninstalled pip-20.2b1**
**Using virtual environment 'python_environment' ...**
**Collecting pip==19.0.3**
**Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)**
**Collecting pandas**
**Downloading https://files.pythonhosted.org/packages/53/87/6438c197fc70ca6b3056cfb60b3dfedca25bedb631bce1f72d6a10502d15/pandas-1.0.4.tar.gz (5.0MB)**
**Complete output from command python setup.py egg_info:**
**Traceback (most recent call last):**
**f"numpy >= {min_numpy_ver}",**
**                            ^**
**You are using pip version 8.1.1, however version 20.1.1 is available.**
**You should consider upgrading via the 'pip install --upgrade pip' command.**
**SyntaxError: invalid syntax**
** ----------------------------------------**
**Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-udl4526_/pandas/**
**Warning: Error in : Error installing package(s): 'pip==19.0.3', 'langdetect', 'pandas', 'numpy', 'googletrans', 'xlrd'**
**File "<string>", line 1, in <module>**
**File "/tmp/pip-build-udl4526_/pandas/setup.py", line 42**
**63: pip_install**
**61: server [/srv/connect/apps/xx/app.R#125]**
**62: virtualenv_install**
**64: stop**
**Error : Error installing package(s): 'pip==19.0.3', 'langdetect', 'pandas', 'numpy', 'googletrans', 'xlrd'**

Thanks !

#7

I managed to fix the problem - in case others have a similar issue, I installed an older version of pandas 0.23.3 (compatible with python 3.5):

virtualenv_create(envname = "python_environment", python= "python3")
virtualenv_remove(envname = "python_environment", packages = "pip")
virtualenv_install("python_environment", packages = c('pip==20.0.2', 'langdetect', 'pandas==0.23.3','numpy','googletrans', 'xlrd'), ignore_installed = TRUE)
use_virtualenv("python_environment", required = TRUE)
#8

I solved this today by updating my version of reticulate from 1.15 to 1.16! See discussion here: https://github.com/rstudio/reticulate/issues/757

1 Like