Setting up your own Shiny and RStudio server on a Raspberry Pi

:warning: I have and updated version of this article in my blog Andres' Blog and another article about this topic using a new installation method, you can check it out at Andres' Blog

Si hablas español, puedes encontrar la version en español de esta guía (así como la nueva version en español e ingles) en mi blog El blog de Andrés

I have recently participated in a post that was asking if a Raspberry Pi 3B+ could make it as a viable shiny-server, and was pointed out to me that this might be an interesting topic for some people, so I´m going to share with you my experience setting up a shiny-server on a Raspberry Pi 3B+.

The steps that I'm going to share with you, are applicable for this specific setup, Raspberry Pi 3B+ running Raspbian Stretch Lite (not tested yet in the new Raspbian Buster), and these might not be directly applicable for other SBCs, Linux distributions or even other versions of Raspberry Pi SBCs.

I’m installing all the software directly on the OS (Raspbian), I’m aware that there are other options such as docker images, but this is the method I prefer.

Also, as a disclaimer, I’m assuming some really basic Linux command line skills by the part of the reader.

Preparing the Raspberry Pi

First, we need to start loading a fresh image of Raspbian Stretch Lite onto our SD card (I recommend at list 32GB class 10), you can download it from the official site https://www.raspberrypi.org/downloads/raspbian/.

Although not mandatory, I recommend performing the following actions using the sudo raspi-config application.

  • Expand Filesystem (Advanced Options / Expand Filesystem)
  • Reduce GPU memory to 16MB (Advanced Options / Memory Split)
  • Disable “Predictable network interfaces” (Network Options / Network interface names)

I also recommend using a wired internet connection and disabling the onboard wifi/Bluetooth module. You can disable it persistently editing this file sudo nano /boot/config.txt by adding these two lines:

dtoverlay=pi3-disable-bt
dtoverlay=pi3-disable-wifi

Next, we need to set a static IP for our raspberry pi, edit this file sudo nano /etc/dhcpcd.conf accordingly with your own needs, this is an example IP configuration.

# Example static IP configuration:
interface eth0
static ip_address=192.168.1.101/24
#static ip6_address=fd51:42f8:caae:d92e::ff/64
static routers=192.168.1.1
static domain_name_servers=192.168.1.1 8.8.8.8 fd51:42f8:caae:d92e::1

Enable all the repositories uncommenting all lines in this file sudo nano /etc/apt/sources.list and then perform the mandatory updates sudo apt update && sudo apt full-upgrade

You have to add at list 1GB of swap memory for this installation but you are going to need more, later for installing/compiling some R libraries, so with 3GB would be fine, you can do it with these commands.

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=3072
sudo /sbin/mkswap /var/swap.1
sudo /sbin/swapon /var/swap.1
sudo sh -c 'echo "/var/swap.1 swap swap defaults 0 0 " >> /etc/fstab'

Prevent the unnecessary use of swap memory for protecting your SD card

sudo nano /etc/sysctl.conf
# Add this at the end
    vm.swappiness=10

Setting up the Server

For convenience, we need an HTML server and Nginx is a really good option, fort its reduced requirements, you can install Nginx with these commands:

sudo apt install nginx
sudo chown -R www-data:pi /var/www/html/
sudo chmod -R 770 /var/www/html/

Then we need a sql server for storing and manipulating our data, I prefer PostgreSQL since it’s open-source and has impressive capabilities, but it´s up to you, you can install PostgreSQL with this command.

sudo apt install postgresql libpq-dev postgresql-client postgresql-client-common

Configure postgresql to accept local and remote connections by editing these files and modify/add/comment these lines:

sudo nano /etc/postgresql/9.6/main/pg_hba.conf # Edit as it follows
    local all all md5 #Modify 
    host all all 192.168.1.0/24 trust #Add
    host all all 0.0.0.0/0 password #Add

sudo nano /etc/postgresql/9.6/main/postgresql.conf # Edit as it follows
    listen_addresses #Comment 
    listen_addresses='*' #Add 

Restart postgresql service

sudo systemctl restart postgresql

Create the pi user and database in PostgreSQL server

sudo su postgres
createuser pi -P --interactive
psql
create database pi;
\q
exit

Now, we get to the R specific part of the process, we are going to install the latest version of R from source with these commands:

sudo apt-get install -y gfortran libreadline6-dev libx11-dev libxt-dev \
                               libpng-dev libjpeg-dev libcairo2-dev xvfb \
                               libbz2-dev libzstd-dev liblzma-dev \
                               libcurl4-openssl-dev \
                               texinfo texlive texlive-fonts-extra \
                               screen wget libpcre2-dev
cd /usr/local/src
sudo wget https://cran.rstudio.com/src/base/R-4/R-4.0.2.tar.gz
sudo su
tar zxvf R-4.0.2.tar.gz
cd R-4.0.2
./configure --enable-R-shlib #--with-blas --with-lapack #optional
make
make install
cd ..
rm -rf R-4.0.2*
exit
cd

WARNING: EVERYTHING BEYOND THIS POINT IS EXTREMELY TIME CONSUMING
e.g. Installing RStudio from source on Raspbian can take around 27 hours :open_mouth:

Install Shiny-Server

We are going to install shiny-server from source, but first, we have to install its dependencies. (It's necessary to install R packages as superuser, for shiny to bean able to use them).

sudo su - -c "R -e \"install.packages('later', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('fs', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('Rcpp', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('httpuv', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('mime', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('jsonlite', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('digest', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('htmltools', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('xtable', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('R6', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('Cairo', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('sourcetools', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('shiny', repos='http://cran.rstudio.com/')\""

Install cmake from source

wget https://cmake.org/files/v3.17/cmake-3.17.0.tar.gz # Buscar la ultima version en https://cmake.org/files/
tar xzf cmake-3.17.0.tar.gz
cd cmake-3.17.0
./configure; make
sudo make install
cd
rm cmake-3.17.0.tar.gz
rm -rf cmake-3.17.0

Install shiny-server

git clone https://github.com/rstudio/shiny-server.git
cd shiny-server
DIR=`pwd`
PATH=$DIR/bin:$PATH
mkdir tmp
cd tmp
PYTHON=`which python`
sudo cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DPYTHON="$PYTHON" ../
sudo make
mkdir ../build
sed -i '8s/.*/NODE_SHA256=a865e69914c568fcb28be7a1bf970236725a06a8fc66530799300181d2584a49/' ../external/node/install-node.sh # node-v12.15.0-linux-armv7l.tar.xz
sed -i 's/linux-x64.tar.xz/linux-armv7l.tar.xz/' ../external/node/install-node.sh
sed -i 's/https:\/\/github.com\/jcheng5\/node-centos6\/releases\/download\//https:\/\/nodejs.org\/dist\//' ../external/node/install-node.sh
(cd .. && sudo ./external/node/install-node.sh)
(cd .. && ./bin/npm --python="${PYTHON}" install --no-optional)
(cd .. && ./bin/npm --python="${PYTHON}" rebuild)
sudo make install

Configure shiny-server

cd
sudo ln -s /usr/local/shiny-server/bin/shiny-server /usr/bin/shiny-server
sudo useradd -r -m shiny
sudo mkdir -p /var/log/shiny-server
sudo mkdir -p /srv/shiny-server
sudo mkdir -p /var/lib/shiny-server
sudo chown shiny /var/log/shiny-server
sudo mkdir -p /etc/shiny-server
cd
sudo wget \
https://raw.github.com/rstudio/shiny-server/master/config/upstart/shiny-server.conf \
-O /etc/init/shiny-server.conf
sudo chmod 777 -R /srv
# Configure shiny-server autostart 
sudo nano /lib/systemd/system/shiny-server.service # Paste the following
    #!/usr/bin/env bash
    [Unit]
    Description=ShinyServer
    [Service]
    Type=simple
    ExecStart=/usr/bin/shiny-server
    Restart=always
    # Environment="LANG=en_US.UTF-8"
    ExecReload=/bin/kill -HUP $MAINPID
    ExecStopPost=/bin/sleep 5
    RestartSec=1
    [Install]
    WantedBy=multi-user.target

sudo chown shiny /lib/systemd/system/shiny-server.service
sudo systemctl daemon-reload
sudo systemctl enable shiny-server
sudo systemctl start shiny-server

Setting up proper user permissions

sudo groupadd shiny-apps
sudo usermod -aG shiny-apps pi
sudo usermod -aG shiny-apps shiny
cd /srv/shiny-server
sudo chown -R pi:shiny-apps .
sudo chmod g+w .
sudo chmod g+s .

Install Rstudio

We are going to install RStudio server from source, this is extremely time-consuming (around 27 hours last time I checked), so get your self something to do while the raspberry pi installs dependencies and compiles everything.

# Install Rust and Cargo
sudo curl https://sh.rustup.rs -sSf | sh
source $HOME/.cargo/env
# Install sentry-cli
sudo git clone https://github.com/getsentry/sentry-cli.git
sudo chown -R pi ~/sentry-cli
cd sentry-cli
cargo build
cd
sudo rm -rf sentry-cli

# Clone RStudio repository
sudo git clone https://github.com/rstudio/rstudio.git

# Avoid installing crashpad
sudo nano /home/pi/rstudio/dependencies/common/install-common
    # Comment these lines
        # ./install-crashpad
        # sudo ./install-crashpad

# Install system dependencies
cd rstudio/dependencies/linux
sudo su
./install-dependencies-stretch --exclude-qt-sdk
apt-get install -y openjdk-8-jdk
apt autoremove

# Configure Java
java -Xms1800m

# Install latest gwt compiler
cd /home/pi/rstudio
wget http://dl.google.com/closure-compiler/compiler-latest.zip
unzip compiler-latest.zip
rm COPYING README.md compiler-latest.zip
mv closure-compiler-v20*.jar /home/pi/rstudio/src/gwt/tools/compiler/compiler.jar

# Compile and install RStudio
mkdir build
cd build
cmake .. -DRSTUDIO_TARGET=Server -DCMAKE_BUILD_TYPE=Release
make install

#Configure Rstudio
cd
useradd -r rstudio-server
cp /usr/local/lib/rstudio-server/extras/init.d/debian/rstudio-server /etc/init.d/rstudio-server
chmod +x /etc/init.d/rstudio-server 
update-rc.d rstudio-server defaults
ln -f -s /usr/local/lib/rstudio-server/bin/rstudio-server /usr/sbin/rstudio-server
chmod 777 -R /usr/local/lib/R/site-library/
mkdir -p /var/run/rstudio-server
mkdir -p /var/lock/rstudio-server
mkdir -p /var/log/rstudio-server
mkdir -p /var/lib/rstudio-server
rm -rf /home/pi/rstudio
nano /etc/init.d/rstudio-server
    # Modify your PATH for using the compiled version of R
        # PATH=/usr/local/bin/:/sbin:/usr/sbin:/bin:/usr/bin
systemctl daemon-reload
rstudio-server start
exit

Extra steps

Make pretty URLs for Shiny and Rstudio Server by editing nginx config (e.g. http://your-ip/shiny/your-app instead of http://your-ip:3838/your-app)

sudo nano /etc/nginx/sites-enabled/default

Add the following lines before the line that reads server {

map $http_upgrade $connection_upgrade {
      default upgrade;
      ''      close;
    }

Add the following lines right after the line that reads server_name _;

    rewrite ^/shiny$ $scheme://$http_host/shiny/ permanent;

    location /shiny/ {
            rewrite ^/shiny/(.*)$ /$1 break;
            proxy_pass http://localhost:3838;
            proxy_redirect / $scheme://$http_host/shiny/;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
            proxy_read_timeout 20d;
            proxy_buffering off;
    }
    
    rewrite ^/rstudio$ $scheme://$http_host/rstudio/ permanent; 
    
    location /rstudio/ {
        rewrite ^/rstudio/(.*)$ /$1 break;
        proxy_pass http://localhost:8787;
        proxy_redirect http://localhost:8787/ $scheme://$http_host/rstudio/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_read_timeout 20d;
    }

Now you have to restart nginx to apply changes

sudo service nginx restart

If you want to have access to your server from remote origins (over the internet) you must follow these extra steps:

  • If you don't have a static public IP

    • Open an account in a Dynamic DNS Services e.g. https://www.noip.com
    • Configure Dynamic DNS in your router/modem, if you don’t have that option, you can install a DDNS client on your server.
    • Optionally configure your personal domain name to point to your DDNS service
  • Configure port forwarding in your router/modem

    • For SSH open port TCP/UDP 22~22
    • For PostgreSQL open port TCP/UDP 5432
    • For FTP open port TCP 20~21
    • For VPN open port UDP 1194
    • For HTTP open port TCP 80
  • Make sure you have “Block WAN traffic” disabled on your router/modem

And this is it, you are ready to start putting your shiny apps inside the /srv/shiny-server folder, enjoy!

Some final comments

The major limitation of the Raspberry Pi is RAM memory, so keep in mind that you have to keep your apps simple enough, so they don’t need more than 1 GB of RAM, if your app requires something greater than that, it's going to be using swap memory and that could ruin your SD card very fast.

The performance is relatively slow, especially for plotting, keep in mind that even though the raspberry pi has 4 cores they are not fast ones and most of the r packages don’t use parallel computing.

P.D. I apologize for all the typos and bad redaction, in my defense, English is not my native language.

31 Likes

This is an awesome post, thank you very much for sharing! I really appreciate that.

I should be sort of angry at you, for you robbed me of much of the joy of discovery, but I can not :wink:

3 Likes

That's absolutely fantastic, thanks so much!

I'll give it a try as soon as time permits.

1 Like

I'm sorry man, I suppose I should include a spoiler alert

1 Like

Thank you so much for sharing!!

1 Like

I have updated the part on installing R, now installs 3.5.2 version

2 Likes

I have added rstudio-server installation section, also I have modified R installation part since R needs to be compiled with --enable-R-shlib option in order for rstudio to work

Looking good so far, cheers mate. By the way, am I going to regret using 15gb, that's all I had lying around?

That depends on your particular application, 15GB it's enough for installing all the basic stuff, although a little short, if you are planning to install other things, like for example, libreoffice for knitting documents to docx

You know what, I had a funny feeling you'd say something like that. lol Sadly this is my first install R and Shiny on pi. Having said that, I now have attempted numerous things, owncloud (didn't work, probably my inexperience), then samba, then went, screw it, I'll install RStudio and check out Shiny apps as I'm learning R stuff for uni. I have been to many sites and tried their install notes only to be disappointed, yours is by far the best I've ever seen! Worst part is I want to ask many questions. :smiley:

Thing is, I have a powered old 750gb hdd hanging off the pi waiting to be used and too poor to buy even a sd card at the moment, so I'll just have some fun and learn some unix/raspbian. You're knowledge of it all is pretty cool and I'll post a proper question if I run into trouble. Cheers again.

Wow! the dedication to put the software together - "12 hours to install RStudio" - Yikes! And I was complaining to myself that I was taking a week to configure my R environment.
You have motivated me to share here and elsewhere my learnings as well. Thank you! I'm not into Raspberry Pi, but I still dig your enthusiasm and dedication.

"file ‘src/Makevars’ has the wrong MD5 checksum" <- I am guessing this is normal, however could you please explain when you have the time. @andresrcs

I don't now exactly in which step you are now but if you have already modified the makevars file its normal that the md5 checksum doesn't match.

If this is your goal, maybe you want to try other options like an AWS virtual server, they offer a free tier that it's obviously more powerful than the pi, and the installation process is much easier since you would be using a regular Linux distribution.

I don't start back at uni till 15th Jan, so have the time to learn something different. Plus, don't like AWS or amazons, avoid it if I can. In the end, it will just be a file server, if I can use if to make a webpage then, YAY, otherwise, not overly bothered.

Yeah, I should have been more specific, but makes sense :+1:

Section installing Rstudio -

CMake Error at src/cpp/session/CMakeLists.txt:23 (message): Dictionaries not found (re-run install-dependencies script to install)

Any clue what I can do to fix this?

Have you installed cmake from source?
Have in mind that the steps are not independent of each other, several of them rely on the previous ones

wget https://cmake.org/files/v3.13/cmake-3.13.0.tar.gz # Check for the latest version on https://cmake.org/files/
tar xzf cmake-3.13.0.tar.gz
cd cmake-3.13.0
./configure; make
sudo make install
cd

Have you installed cmake from source? Yup

When you have run this script did you get any error message?

./install-dependencies-debian --exclude-qt-sdk

Which is the command that you are running when you get the cmake error?