Enabling logging -- RStudio, CentOS 7, and realm/sssd/Active Directory

[edit: added info about /var/lib/rstudio-server]

I have a CentOS 7 system running in AWS, accessed via a Pulse Secure VPN that's using the realm tool to configure access for most of the users. Generally, things work fine.

I've installed rstudio-server.x86_64, version 1.1.463-1 via yum.

I've added an /etc/rstudio/rserver.conf file that sets rsession-which-r to point to our Spack installed version of R.

Initially I was unable to successfully enter a username and password, it responds with

Error: Incorrect or invalid username/password

If I create an /etc/pam.d/rstudio that contains

#%PAM-1.0
auth      requisite      pam_succeed_if.so uid >= 500 quiet
auth      required       pam_sss.so nodelay

account   required       pam_sss.so

then I make it past the login box and get to watch a spinner for a bit before being presented with a dialog box that says:

Unable to connect to service

The javascript console tells me:

WebExtension::executeScript: content script injected
rstudio-0.js:8903 GET http://bifx1n03.bold.bio:8787/mathjax/MathJax.js?config=TeX-MML-AM_CHTML net::ERR_ABORTED 502 (Bad Gateway)
blf @ rstudio-0.js:8903
alf @ rstudio-0.js:1495
PIe @ rstudio-0.js:9980
hCe @ rstudio-0.js:2606
fCe @ rstudio-1.js:1356
Ryi @ rstudio-26.js:4546
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
Pyi @ rstudio-26.js:4546
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
Nyi @ rstudio-26.js:4546
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
Lyi @ rstudio-26.js:4546
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
Jyi @ rstudio-26.js:4546
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
Gyi @ rstudio-26.js:1886
aMj @ rstudio-1.js:1356
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
$Lj @ rstudio-1.js:1356
Pnd @ rstudio-0.js:10195
Jl @ rstudio-0.js:8702
yl @ rstudio-0.js:7186
Nl @ rstudio-0.js:10191
Hl @ rstudio-0.js:1528
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
d @ rstudio-0.js:7953
setTimeout (async)
Kl @ rstudio-0.js:7954
Al @ rstudio-0.js:7424
Fl @ rstudio-0.js:4006
Nnd @ rstudio-0.js:10195
(anonymous) @ rstudio-0.js:8028
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
load (async)
Knd @ rstudio-0.js:8028
Jnd @ rstudio-0.js:8885
qGj @ rstudio-1.js:590
bCe @ rstudio-1.js:1356
zk @ rstudio-0.js:9572
Ek @ rstudio-0.js:572
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
(anonymous) @ rstudio-1.js:1356
rstudio.__installRunAsyncCode @ rstudio.nocache.js:11
__gwtInstallCode @ rstudio-0.js:1
ql @ rstudio-0.js:8744
Rl @ rstudio-0.js:4008
(anonymous) @ rstudio-0.js:6480
_k @ rstudio-0.js:4105
cl @ rstudio-0.js:8761
(anonymous) @ rstudio-0.js:8138
(anonymous) @ 1.cache.js:1
rstudio-0.js:4883 1: Unable to connect to service

If I try that URL directly (http://bifx1n03.bold.bio:8787/mathjax/MathJax.js?config=TeX-MML-AM_CHTML) in the browser I get an ERROR 502 message.

Perhaps that's a red herring...

I can't find any useful logs anywhere, there's nothing relevant in /var/log/messages, and /var/log/rstudio-server is empty. /var/lib/rstudio-server/monitor (as suggested here) does not exist.

I'd appreciate any suggestions about how to proceed.

Thanks!

g.

Got it. Used verify-installation for the first hint:

[ghartzell@spiffy1n03 bin]$ sudo rstudio-server verify-installation
/usr/lib/rstudio-server/bin/rsession: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-5.5.0/icu4c-60.1-5jsoxxlzmwbcamsrsoq53k5gf4dm2glx/lib/libicuuc.so.60)
/usr/lib/rstudio-server/bin/rsession: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-5.5.0/icu4c-60.1-5jsoxxlzmwbcamsrsoq53k5gf4dm2glx/lib/libicui18n.so.60)

Then checked what was up with the icu library:

[ghartzell@spiffy1n03 bin]$ ldd /spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-5.5.0/icu4c-60.1-5jsoxxlzmwbcamsrsoq53k5gf4dm2glx/lib/libicuuc.so.60
  linux-vdso.so.1 =>  (0x00007fff50bea000)
  libicudata.so.60 =>
/spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-5.5.0/icu4c-60.1-5jsoxxlzmwbcamsrsoq53k5gf4dm2glx/lib/libicudata.so.60
(0x00007f3d64acd000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3d648b1000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f3d646ad000)
  libstdc++.so.6 =>
/spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-5.5.0-uebsji7cdpv74tahlrf27dfk7gzrmgsa/lib64/libstdc++.so.6
(0x00007f3d64331000)
  libm.so.6 => /lib64/libm.so.6 (0x00007f3d6402f000)
  libgcc_s.so.1 =>
/spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-5.5.0-uebsji7cdpv74tahlrf27dfk7gzrmgsa/lib64/libgcc_s.so.1
(0x00007f3d63e18000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f3d63a4b000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f3d66a18000)

Note that the verify-installation output is grabbing the C++ library from /lib64, but the icu lib is using (via RPATH) the one that Spack used to build it (from the gcc@5.5.0 tree).

I already had an entry in /etc/rstudio/rserver.conf to use the R I'd built via Spack. Added an entry to the library search path so that the icu library above would find the C++ library it was looking for et voila!

Here's the rserver.conf that's working (so far):

rsession-which-r=/spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-5.5.0/r-3.5.1-pvmnul3uhaq6eyyrdnfz36g5ooefnnry/bin/R
rsession-ld-library-path=/spiffy/apps/spack/spack-20181217/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-5.5.0-uebsji7cdpv74tahlrf27dfk7gzrmgsa/\
lib64/

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

My original bit of question stands though. Why wasn't anything useful being logged anywhere when this was blowing up?