Incompatibilities with Python Installation and OpenSSL

[hansen_m]

I've configured RStudio Server, multiple R installations and Python 3 according to the best practices listed in the documentation. One issue I'm running into is I believe incompatibilities with OpenSSL are causing issues with things like scp and Git via ssh.

When I open a new session my LD_LIBRARY_PATH looks like this:

LD_LIBRARY_PATH=/opt/R/4.0.5/lib/R/lib::/lib:/usr/local/lib:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/amd64/server

If I open a .py file, and execute it via Reticulate it executes fine but opening a new Terminal window my LD_LIBRARY_PATH looks like this:

LD_LIBRARY_PATH=/opt/Python/3.9.1/lib:/opt/Python/3.9.1/lib:/opt/Python/3.9.1/lib:/opt/Python/3.9.1/lib:/opt/Python/3.9.1/lib:/opt/Python/3.9.1/lib:/opt/R/4.0.5/lib/R/lib::/lib:/usr/local/lib:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-1.el8_3.x86_64/jre/lib/amd64/server

When attempting to scp or Git clone with ssh I see errors like:

/usr/bin/ssh: symbol lookup error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b
lost connection

Thoughts or advice on better ways to support this?

Thanks!

[rstub]

Which OS are you using? What is the output of ldd /usr/bin/ssh in the two terminals?

[hansen_m]

CentOS Linux release 8.3.2011

R

$ ldd /usr/bin/ssh
        linux-vdso.so.1 (0x00007ffea8da6000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fc1e5d70000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fc1e5b6c000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007fc1e5968000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fc1e5751000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fc1e5528000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fc1e5311000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fc1e50e7000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fc1e4e92000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fc1e4ba9000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fc1e4992000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fc1e478e000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fc1e43cb000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc1e41ab000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc1e6513000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007fc1e3f27000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fc1e3d16000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fc1e3b12000)

Python

$ ldd /usr/bin/ssh
        linux-vdso.so.1 (0x00007ffe6f4c9000)
        libcrypto.so.1.1 => /opt/Python/3.9.1/lib/libcrypto.so.1.1 (0x00007f973fe76000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f973fc72000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007f973fa6e000)
        libz.so.1 => /opt/Python/3.9.1/lib/libz.so.1 (0x00007f97405fd000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f973f845000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f973f62e000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f973f404000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f973f1af000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f973eec6000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f973ecaf000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f973eaab000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f973e6e8000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f973e4c8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f97403ff000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f973e244000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f973e033000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f973de2f000)

I believe the issue is the change to libcrypto.so.1.1 => /opt/Python/3.9.1 but unsure of how to address this without potentially breaking other functionality.

[rstub]

Indeed, it looks like the binary python installation comes with its own version of OpenSSL that is incompatible with the system version, even though both are version 1.1. Maybe they have been compiled with slightly different options.

Can you try installing Python from source as a workaround? RStudio Install Python From Source - RStudio Documentation

[hansen_m]

I can give that a try. Think that would solve it?

These are the steps I used, with miniconda - Integrating RStudio Server Pro with Python - RStudio Documentation

Any downside as far as RStudio integration/support?

[hansen_m]

Installing from source seemed to work just fine. I'll have to do some more testing and have some end users try it out but seems to be a workable solution. Anaconda environments would be more flexible will give this a shot. Thank you, @rstub!

[rstub]

Great to hear that! When compiling from source you will use things like OpenSSL from the system, keeping it consistent with other tools. Unfortunately python from miniconda includes some of these libraries, resulting in the incompatibilities you have encounterted.