Setting up your own shiny-server on a Raspberry Pi 3B+

rasperry-pi

#1

I have recently participated on a post that was asking if a Raspberry Pi 3B+ could make it as a viable shiny-server, and was pointed out to me that this might be an interesting topic for some people, so I´m going to share with you my experience setting up a shiny-server on a Raspberry Pi 3B+.

The steps that I'm going to share with you, are applicable for this specific setup, Raspberry Pi 3B+ running Raspbian Stretch Lite (we have to reserve all possible resources for the shiny-server, so graphical interface is a no go for this application), these might not be directly applicable for other SBCs, Linux distributions or even other versions of Raspberry Pi SBCs.

I’m installing all the software directly on the OS (Raspbian), I’m aware that there are other options such as doker images, but this is the method I prefer.

Also, as a disclaimer, I’m assuming some really basic Linux command line skills by the part of the reader.

Preparing the Raspberry Pi

First we need to start loading a fresh image of Raspbian Stretch Lite into our SD card (I recommend at list 32GB class 10), you can download it from the official site https://www.raspberrypi.org/downloads/raspbian/.

Although not mandatory, I recommend performing the following actions using the sudo raspi-config application.

  • Expand Filesystem (Advanced Options / Expand Filesystem)
  • Reduce GPU memory to 16MB (Advanced Options / Memory Split)
  • Disable “Predictable network interfaces” (Network Options / Network interface names)

I also recommend using a wired internet connection and disabling the onboard wifi/Bluetooth module. You can disable it persistently editing this file sudo nano /boot/config.txt by adding these two lines:

dtoverlay=pi3-disable-bt
dtoverlay=pi3-disable-wifi

Next, we need to set a static IP for our raspberry pi, edit this file sudo nano /etc/dhcpcd.conf accordingly to your own needs, this is an example IP configuration.

# Example static IP configuration:
interface eth0
static ip_address=192.168.1.101/24
#static ip6_address=fd51:42f8:caae:d92e::ff/64
static routers=192.168.1.1
static domain_name_servers=192.168.1.1 8.8.8.8 fd51:42f8:caae:d92e::1

Enable all the repositories uncommenting all lines in this file sudo nano /etc/apt/sources.list and then perform the mandatory updates sudo apt update && sudo apt full-upgrade

You have to add at list 1GB of swap memory for this installation but you are going to need more later for installing/compiling some R libraries, so with 3GB would be fine, you can do it with these commands.

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=3072
sudo /sbin/mkswap /var/swap.1
sudo /sbin/swapon /var/swap.1
sudo sh -c 'echo "/var/swap.1 swap swap defaults 0 0 " >> /etc/fstab'

Setting up the Server

For convenience, we need a html server and nginx is a really good option, fort its reduced requirements, you can install nginx with theses commands:

sudo apt install nginx
sudo chown -R www-data:pi /var/www/html/
sudo chmod -R 770 /var/www/html/

Then we need a sql server for storing and manipulating our data, I prefer postgresql since it’s open source and has impressive capabilities, but it´s up to you, you can install postgresql with this command.

sudo apt install postgresql libpq-dev postgresql-client postgresql-client-common

Configure postgresql to accept local and remote connections by editing these files and modify/add/comment these lines:

sudo nano /etc/postgresql/9.6/main/pg_hba.conf # Edit as it follows
    local all all md5 #Modify 
    host all all 192.168.1.0/24 trust #Add
    host all all 0.0.0.0/0 password #Add

sudo nano /etc/postgresql/9.6/main/postgresql.conf # Edit as it follows
    listen_addresses #Comment 
    listen_addresses='*' #Add 

Restart postgresql service

sudo systemctl restart postgresql

Create the pi user and database in postgresql server

sudo su postgres
createuser pi -P --interactive
psql
create database pi;
\q
exit

Now, we get to the R specific part of the process, we are going to install R 3.5.1 from source with these commands:

sudo apt-get install -y gfortran libreadline6-dev libx11-dev libxt-dev \
libpng-dev libjpeg-dev libcairo2-dev xvfb \
libbz2-dev libzstd-dev liblzma-dev \
libcurl4-openssl-dev \
texinfo texlive texlive-fonts-extra \
screen wget

cd /usr/local/src
sudo wget https://cran.rstudio.com/src/base/R-3/R-3.5.1.tar.gz
sudo su -
cd /usr/local/src
tar zxvf R-3.5.1.tar.gz
cd R-3.5.1

Before continuing to the compile and install part we have to modify this file sudo nano ./configure by changing :: to ;; on line 27483. Then we continue with:

./configure
make
make install

And now the most important and time-consuming part of the process, installing shiny-server from source.

We have to install it’s dependencies, some R libraries present problems installing on an armhf architecture so we are going to address those first (by the time you are reading, this might be no longer necessary, check the development versions of the libraries first).

Install fs package from source

cd ~
wget https://cran-r.c3sl.ufpr.br/src/contrib/fs_1.2.6.tar.gz
tar zxvf fs_1.2.6.tar.gz
sudo nano fs/src/Makevars # Edit this file
    ifeq ($(UNAME), Linux)
        PKG_LIBS += -pthread # Add this line here
        OBJECTS += bsd/setmode.o bsd/strmode.o bsd/reallocarray.o
    endif
sudo R
install.packages('fs', repos = NULL, type='source')

Install later package from source

sudo apt-get install libboost-atomic-dev
git clone https://github.com/r-lib/later.git
sudo nano later/src/Makevars.in #changed PKG_LIBS = -pthread @libs@ with PKG_LIBS = -pthread -lboost_atomic @libs@
sudo R CMD INSTALL later

Install the rest of it’s dependencies (Is necessary to install r packages as super user, for shiny to bean able to use them).

sudo su - -c "R -e \"install.packages('Rcpp', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('httpuv', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('mime', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('jsonlite', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('digest', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('htmltools', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('xtable', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('R6', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('Cairo', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('sourcetools', repos='http://cran.rstudio.com/')\""
sudo su - -c "R -e \"install.packages('shiny', repos='http://cran.rstudio.com/')\""

Install cmake from source

wget https://cmake.org/files/v3.13/cmake-3.13.0.tar.gz # Check for the latest version on https://cmake.org/files/
tar xzf cmake-3.13.0.tar.gz
cd cmake-3.13.0
./configure; make
sudo make install
cd

Install shiny-server

git clone https://github.com/rstudio/shiny-server.git
cd shiny-server
DIR=`pwd`
PATH=$DIR/bin:$PATH
mkdir tmp
cd tmp
PYTHON=`which python`
sudo cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DPYTHON="$PYTHON" ../
sudo make
mkdir ../build
sed -i '8s/.*/NODE_SHA256=7a2bb6e37615fa45926ac0ad4e5ecda4a98e2956e468dedc337117bfbae0ac68/' ../external/node/install-node.sh
sed -i 's/linux-x64.tar.xz/linux-armv7l.tar.xz/' ../external/node/install-node.sh
(cd .. && sudo ./external/node/install-node.sh)
(cd .. && ./bin/npm --python="${PYTHON}" install --no-optional)
(cd .. && ./bin/npm --python="${PYTHON}" rebuild)
sudo make install

Configure shiny-server

cd
sudo ln -s /usr/local/shiny-server/bin/shiny-server /usr/bin/shiny-server
sudo useradd -r -m shiny
sudo mkdir -p /var/log/shiny-server
sudo mkdir -p /srv/shiny-server
sudo mkdir -p /var/lib/shiny-server
sudo chown shiny /var/log/shiny-server
sudo mkdir -p /etc/shiny-server
cd
sudo wget\
https://raw.github.com/rstudio/shiny-server/master/config/upstart/shiny-server.conf\
-O /etc/init/shiny-server.conf
sudo chmod 777 -R /srv
# Configure shiny-server autostart 
sudo nano /lib/systemd/system/shiny-server.service # Paste the following
    #!/usr/bin/env bash
    [Unit]
    Description=ShinyServer
    [Service]
    Type=simple
    ExecStart=/usr/bin/shiny-server
    Restart=always
    # Environment="LANG=en_US.UTF-8"
    ExecReload=/bin/kill -HUP $MAINPID
    ExecStopPost=/bin/sleep 5
    RestartSec=1
    [Install]
    WantedBy=multi-user.target

sudo chown shiny /lib/systemd/system/shiny-server.service
sudo systemctl daemon-reload
sudo systemctl enable shiny-server
sudo systemctl start shiny-server

Setting up proper user permissions

sudo groupadd shiny-apps
sudo usermod -aG shiny-apps pi
sudo usermod -aG shiny-apps shiny
cd /srv/shiny-server
sudo chown -R pi:shiny-apps .
sudo chmod g+w .
sudo chmod g+s .

Extra steps

Make pretty URL for Shiny Server by editing nginx config (http://your-ip/shiny/your-app instead of http://your-ip:3838/your-app)

sudo nano /etc/nginx/sites-enabled/default
# Add the following lines right after the line that reads server_name _;
    location /shiny/ {
        proxy_pass http://127.0.0.1:3838/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        rewrite ^(/shiny/[^/]+)$ $1/ permanent;
    }
sudo service nginx restart

If you want to have access to your server from remote origins (over the internet) you must follow these extra steps:

  • If you don't have a static public IP

    • Open an account in a Dynamic DNS Services e.g. https://www.noip.com
    • Configure Dynamic DNS in your router/modem, if you don’t have that option, you can install a DDNS client on your server.
    • Optionally configure your personal domain name to point to your DDNS service
  • Configure port forwarding in your router/modem

    • For SSH open port TCP/UDP 22~22
    • For PostgreSQL open port TCP/UDP 5432
    • For FTP open port TCP 20~21
    • For VPN open port UDP 1194
    • For HTTP open port TCP 80
  • Make sure you have “Block WAN traffic” disabled on your router/modem

And this is it, you are ready to start putting your shiny apps inside the /srv/shiny-server folder, enjoy!

Some final comments

The major limitation of the Raspberry Pi is RAM memory, so keep in mind that you have to keep your apps simple enough, so they don’t need more than 1 GB of RAM, if your app requires something greater than that, it's going to be using swap memory and that could ruin your SD card very fast.

The performance is relatively slow, especially for plotting, keep in mind that even though, the raspberry pi has 4 cores they are not fast ones and most of the r packages don’t use parallel computing.

P.D. I apologies for all the typos and bad redaction, in my defense, English is not my native language.


Raspberry Pi as a viable Shiny server
#2

This is an awesome post, thank you very much for sharing! I really appreciate that.

I should be sort of angry at you, for you robbed me of much of the joy of discovery, but I can not :wink:


#3

That's absolutely fantastic, thanks so much!

I'll give it a try as soon as time permits.


#4

I'm sorry man, I suppose I should include a spoiler alert


#5

Thank you so much for sharing!!