rserver[1692]: ERROR system error 13 (Permission denied)

[ArchosaurRise]

I am running rstudio-server on Google Cloud Platform VM instance. My OS is CentOS 8. I try to access the rstudio-server using my Firefox browser at the IP address: http://34.94.188.231:8787/ I create a test user called "testuser" and according password called "testpassword".

Then I got a "Rstudio Initialization Error" "Unable to connect to service".

I check the status of rstudio-server by the command: "sudo rstudio-server status", I got the following message:

● rstudio-server.service - RStudio Server
Loaded: loaded (/etc/systemd/system/rstudio-server.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-12-10 19:29:06 UTC; 13min ago
Process: 1097 ExecStart=/usr/lib/rstudio-server/bin/rserver (code=exited, status=0/SUCCESS)
Main PID: 1133 (rserver)
Tasks: 4 (limit: 22408)
Memory: 62.8M
CGroup: /system.slice/rstudio-server.service
├─1133 /usr/lib/rstudio-server/bin/rserver
└─1692 /usr/lib/rstudio-server/bin/rserver

Dec 10 19:29:05 third-test systemd[1]: Starting RStudio Server...
Dec 10 19:29:06 third-test systemd[1]: Started RStudio Server.
Dec 10 19:30:28 third-test rserver[1692]: ERROR system error 13 (Permission denied); OCCURRED AT: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType*) /var/lib/jenkins/workspace/IDE/open-source-pipeline/v1.2-patch/src/cpp/core/system/PosixSystem.cpp:2152; LOGGED FROM: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType*) /var/lib/jenkins/workspace/IDE/open-source-pipeline/v1.2-patch/src/cpp/core/system/PosixSystem.cpp:2153

Can any one help on that? I searched a lot using Google, and tried many suggested suggestions, and it doesn't work.

[cole]

Hello! Apologies for the trouble! Typically the "ERROR system error 13 (Permission denied); " error suggests that the user you are using either:

• does not have a home directory
• does not have write access to the home directory

Can you verify the home directory for your user by SSHing into the server?

getent passwd myusername

And then look at the permissions on that directory?

namei -l /path/to/my/home/directory
ls -lha /path/to/my/home/directory

[kevinushey]

This error:

Dec 10 19:30:28 third-test rserver[1692]: ERROR system error 13 (Permission denied) ; OCCURRED AT: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType*) /var/lib/jenkins/workspace/IDE/open-source-pipeline/v1.2-patch/src/cpp/core/system/PosixSystem.cpp:2152; LOGGED FROM: rstudio::core::Error rstudio::core::system::launchChildProcess(std::__cxx11::string, std::__cxx11::string, rstudio::core::system::ProcessConfig, rstudio::core::system::ProcessConfigFilter, PidType*) /var/lib/jenkins/workspace/IDE/open-source-pipeline/v1.2-patch/src/cpp/core/system/PosixSystem.cpp:2153

is being emitted here:

rstudio/rstudio/blob/330255ddec489e7a147ace3e8a9a3e4157d8d5ad/src/cpp/core/system/PosixSystem.cpp#L2148-L2155

which means the call to ::setpgid(0, 0) is failing, for some reason.

[cole]

I noticed some SELinux stuff in your messages file, although little else seemed interesting. Is it possible that SELinux is causing trouble? Can you turn off SELinux and see if that resolves the issue?

I have no idea why the ::setpgid(0, 0) call would be failing (as @kevinushey noted)

[ArchosaurRise]

Hi, Cole and kevinushey,

Thanks for providing suggestions. I disabled the SELinux and rebooted the operating system, now the rstudio-server can be accessed through web browser! Here is the screenshot of success!

Since the SELinux is turned on by default on RedHat/CentOS/Fedora or several derivatives, perhaps you guys can try to make rstudio-server compatible with SELinux, or give a warning in the installation instructions. That would be very helpful for users.

Here is the test process. I refer to the web page here: https://www.tecmint.com/disable-selinux-in-centos-rhel-fedora/ , I tested the hypothesis about SELinux.

[testuser@third-test ~]$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      31

I changed the directive SELinux=enforcing to SELinux=disabled

[testuser@third-test ~]$ sudo vi /etc/sysconfig/selinux

Then I reboot my system.

[testuser@third-test ~]$ sudo shutdown -r now

Now the SELinux is disabled.

[testuser@third-test ~]$ sestatus
SELinux status:                 disabled

Then the rstudio-server status looks good.

[testuser@third-test ~]$ sudo rstudio-server status
● rstudio-server.service - RStudio Server
   Loaded: loaded (/etc/systemd/system/rstudio-server.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-12-12 21:42:54 UTC; 2min 0s ago
  Process: 1091 ExecStart=/usr/lib/rstudio-server/bin/rserver (code=exited, status=0/SUCCESS)
 Main PID: 1135 (rserver)
​    Tasks: 3 (limit: 22408)
   Memory: 58.7M
   CGroup: /system.slice/rstudio-server.service
​           └─1135 /usr/lib/rstudio-server/bin/rserver

Dec 12 21:42:54 third-test systemd[1]: Starting RStudio Server...
Dec 12 21:42:54 third-test systemd[1]: Started RStudio Server.
[testuser@third-test ~]$ sudo rstudio-server status
● rstudio-server.service - RStudio Server
   Loaded: loaded (/etc/systemd/system/rstudio-server.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-12-12 21:42:54 UTC; 3min 37s ago
  Process: 1091 ExecStart=/usr/lib/rstudio-server/bin/rserver (code=exited, status=0/SUCCESS)
 Main PID: 1135 (rserver)
​    Tasks: 8 (limit: 22408)
   Memory: 141.0M
   CGroup: /system.slice/rstudio-server.service
​           ├─1135 /usr/lib/rstudio-server/bin/rserver
​           └─1662 /usr/lib/rstudio-server/bin/rsession -u testuser2 --launcher-token A8F380C6

Dec 12 21:42:54 third-test systemd[1]: Starting RStudio Server...
Dec 12 21:42:54 third-test systemd[1]: Started RStudio Server.

[cole]

Woohoo!! So glad that you got things working! This is definitely on our list of things to tackle, we just have not done so yet:

There is an open github issue here:

RStudio Server does not work with SELinux

And a community thread on the topic here, that I have unfortunately not had much time to dig further into:

SELinux errors with RStudio Server

Thanks for reopening this thread @cole. Since the solution posted on github and stackoverflow is just a workaround, there are two main goals to getting a permanent solution: Get RStudio Server working out of the box with SELinux. The source of the problem is that RStudio Server is packaged with binaries in a library directory usr/lib. This results in binaries having the SELinux type lib_t instead of bin_t. Not following Filesystem Hierarchy Standards is the cause of the SELinux errors. The solution might be as simple as to install the binaries found in /usr/lib/rustudio-server/bin into /usr/bin or /usr/sbin or /usr/libexec as needed. Create a targeted SELinux policy for RStudio Serv…

[ArchosaurRise]

Thanks for the help. Hopefully, the feature could be fixed soon.

[ArchosaurRise]

Hi, Cole,

Thanks for the suggestions. Here is the test results:

[testuser@third-test ~]$ getent passwd testuser
testuser:x:1002:1003::/home/testuser:/bin/bash

[testuser@third-test ~]$ namei -l /home/testuser/
f: /home/testuser/
dr-xr-xr-x root root /
drwxr-xr-x root root home
drwx------ testuser testuser testuser

[testuser@third-test ~]$ ls -lha /home/testuser/
total 16K
drwx------. 3 testuser testuser 95 Dec 10 19:27 .
drwxr-xr-x. 5 root root 58 Dec 9 23:22 ..
-rw-------. 1 testuser testuser 392 Dec 11 00:11 .bash_history
-rw-r--r--. 1 testuser testuser 18 Oct 1 13:26 .bash_logout
-rw-r--r--. 1 testuser testuser 141 Oct 1 13:26 .bash_profile
-rw-r--r--. 1 testuser testuser 312 Oct 1 13:26 .bashrc
drwx------. 2 testuser testuser 29 Dec 10 19:13 .ssh

[cole]

Thanks for sharing! That's interesting! Permissions look fine there. It might be worth clearing the cookies in your browser and then authenticating to RStudio Server again to be sure the issue is the same.

Meanwhile, you should tail -f /var/log/syslog or tail -f /var/log/messages (depending on which exists) to see if you get the same error message.

Are you using the open source version of RStudio Server? You could also try creating a new user to be sure things failing are not because of strange state with the testuser:

useradd -m -s /bin/bash testuser2
passwd testuser2

[cole]

Ahh sad! Sorry to hear about that! At least the problem is reproducible

On the log file, can you try sudo cat /var/log/messages? It should give you a bunch of output.

Can you also share the output of:

namei -l /var/lib/rstudio-server/shared-storage/sessions
ls -lha /var/lib/rstudio-server/shared-storage/sessions

It is very curious that this error is appearing for a user who has a home directory with proper privileges!

[ArchosaurRise]

Hi, Cole,
Thanks for your quick response.

I cleared the cookies from my firefox browser on Windows 7, and tried to log into the Rstudio server again. It popped up the same error again.

Yes. I am using the open source version of Rstudio server. I followed the instructions here:

Download RStudio Server for Red Hat/CentOS

Here is the test result following your suggestions:

[testuser@third-test ~]$ tail -f /var/log/syslog
tail: cannot open '/var/log/syslog' for reading: No such file or directory
tail: no files remaining
[testuser@third-test ~]$ tail -f /var/log/messages
tail: cannot open '/var/log/messages' for reading: Permission denied
tail: no files remaining
[testuser@third-test ~]$ useradd -m -s /bin/bash testuser2
useradd: Permission denied.
useradd: cannot lock /etc/passwd; try again later.
[testuser@third-test ~]$ sudo useradd -m -s /bin/bash testuser2
[testuser@third-test ~]$ sudo passwd testuser2
Changing password for user testuser2.
New password:
BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word
Retype new password:
passwd: all authentication tokens updated successfully.

As you can see, I created a new user called "testuser2" with a password "testpassword2", however, when I logged into rstudio server using my firefox web browser, it popped the same error.

[ArchosaurRise]

Hi, Cole,

Thanks for problem-solving.

Here is the log file generated by sudo cat /var/log/messages:

var_log_messages_for_rstudio.txt

[testuser@third-test ~]$ namei -l /var/lib/rstudio-server/shared-storage/sessions
f: /var/lib/rstudio-server/shared-storage/sessions
dr-xr-xr-x root root /
drwxr-xr-x root root var
drwxr-xr-x root root lib
drwxrwxrwx root root rstudio-server
                     shared-storage - No such file or directory
[testuser@third-test ~]$ ls -lha /var/lib/rstudio-server/shared-storage/sessions
ls: cannot access '/var/lib/rstudio-server/shared-storage/sessions': No such file or directory