Best Practices: Shiny Development


#1

I thought it might be useful to start a thread dedicated to shiny workflow and effective practices.

It would be interesting to hear other people’s thoughts on things like file setup (app.R vs ui.R + server.R + global.R) and more general advice on how you organise/navigate through your code.

As apps grow larger and larger, I find it can be quite ‘fiddly’ to add/remove elements efficiently and a lot of time is spent locating chunks of code or searching for orphan commas and brackets that are causing errors.

It looks like using Shiny Modules would be a good solution for making larger apps more manageable and reproducible but I have yet to test it out.

Thanks!


#2

I’ve made very very few apps (<15), but have used both app.R and ui.R+server.R+global.R.

For me, it is generally easier to just use app.R if the app I’m creating is smaller in size and easily navigated. For my larger projects, I like to separate the code out for easier navigation. For these types of projects, I typically include a run.R file as well that I can use to create a desktop icon or something similar so that I don’t need to open R to run them.


#3

I am partial to the the ui.r/server.r/global.r set up.

Most (all) of my apps are based on the magnificent shiny dashboard package (https://rstudio.github.io/shinydashboard/). A tip that I was told (pre-Shiny modules) was to use functions to keep the UI easy to manage (https://groups.google.com/forum/#!msg/shiny-discuss/-Ym6buKaXRU/AXna6enIOGgJ;context-place=forum/shiny-discuss) - I use this all the time!


#4

My workflow is like below:

  1. Plan out the type of app. For a lot of applications, sometimes a htmlwidget in Rmd will do, and covers more cases as time goes on and javascript gets cooler. This is preferable as then you can publish to any server, not needing Shiny. But if it needs more server side interaction, I look at using flexdashboard to keep Rmd elements, but have shiny embedded. If its a big app though, its easier to keep track using a ui.R or a HTML template. I have a HTML template based on Gentella which I have tweaked for a company template, or if a client wants something embedded in their website make a new HTML template with their CSS, or finally if none of those are appropriate go to using an ui.R file.
  2. I use a separate ui.R and server.R file, no global.R just source supporting scripts at the top or both ui.R and server.R - (source("functions.R")) No big reason why not to use global.R but I potentially change the functions.R into a small package later that supports modules.
  3. The server.R I template with:
library(shiny)
source("functions.R")

function(input, output, session){

}
  1. I then work on the ui.R until I have a UI I like. Add any boilerplate such as tracking and shinyjs if any of those features are useful (love the visible/non-visible flags it has). It can be really hard to debug a ui.R if you get a bracket or comma in the wrong place, so I try to comment the end of big UI tags such as fluidRows() etc. As I’m adding outputX elements that are named I start adding the renderX elements to server.R to keep track of the reactive names.

  2. ui.R Can be annoying when removing elements with trailing commas breaking stuff, so I try to put a br() or div() at the end of each so its easier to delete stuff. I’m also strict with code spacing, put one element per line and run it through CMD+I to auto-indent as often as possible. It can lead to very long lines :-/ Another tip is to use comments at top of major sections, and folding those comments sections in the RStudio interface for sections you are happy with.

  3. Once the ui.R looks reasonable, I first make sure I have a working version of the server side logic outside of shiny, as its a lot easier to debug it there. Then I look to make some reactives() and eventReactives() as needed. I typically have a raw data reactive, then a processed data reactive that uses req() to depend on the raw data and the inputs, then maybe a final plot reactive if used by lots of output elements.

  4. I try to ensure a req() or validate(need() is on every function, and always use NULL when data is invalid within functions.

  5. Then start building the renderOutputs(), ideally keeping the code within simple as other logic is outside. Start running the app locally and checking how they look in the ui.R, probably tweak some columns() and widths.

  6. I debug by making sure any reactive objects are assigned to normal R objects after the req() (raw_data <- raw_data()) which I find keeps things simple, and then use browser() and cat() and str() generously to check bits are doing what I expected them to do. The browser() stops the app right at the top of the function, and you can walk through errors and inspect state more easily.

  7. If there is anything I repeat a lot or use again, will look to convert it into a Shiny module, and put it in a package or a script to source via functions.R. I generally try to move all the reactives() to the top of the function, and outputs to the bottom with logic in-between.

  8. Its not over yet as sometimes when you deploy to the server there are issues, for instance if the versions of packages aren’t available on shinyapps.io or your own server. For public apps shinyapps.io is great, for private I use a combo of containerit, Docker and googleComputeEngineR to deploy it to a VM (this workflow) For user authentication I use googleID, which is at the top of the ui.R and server.R before any data is fetched.

HTH - bit more involved than I initially meant to write!
Mark


#5

All of the shiny apps I’m currently maintaining and developing are embedded in packages. The package directory follows the general format of

pkg_name/
    - inst/
        - Application/
            - www/
            - global.R
            - server.R
            - ui.R
    - man
    - R

Each package includes a launch_application function that uses the system.file utility to find the Application folder and start the app. (I’m sure there are better ways of doing this, but my organization as weird technical policies)

This isn’t much different than sourcing a file with all of the utility functions. I like doing it because it encourages me to write documentation for my functions (which I often appreciate when I have to come back to do maintenance or debugging), and it lets me use the devtools package to perform checks and tests. Checks and tests are useful for preventing problems before the app ever gets to production.

I build a lot of UI elements through functions. I should probably use modules more than I do, but I haven’t yet gotten comfortable with designing modules that handle unique and diverse data structures yet.

When actually coding the app, I break the app into functional sections based on the UI. For example, if I have multiple tabs, I will place all of the code in server.R relevant to that tab in one place in the code. My server.R file usually follows the structure

shinyServer(function(input, output, session){
# Global Variables ---------------------------------------
# Reactive Lists ------------------------------------------
# Reactive Values ---------------------------------------
# General Observers -----------------------------------

# Tab 1 -----------------------------------------------------
# Reactive Values ---------------------------------------
# Observers -----------------------------------------------
# Event Observers --------------------------------------
# Output Elements --------------------------------------
# Download Handlers ----------------------------------

# Repeat the above structure for each tab/grouping

# Output Options
})

Clean Up UI Code
Version control with Google Drive
#6

A totally different way to approach large Shiny sites is to abandon menu tabs and use URL-based menus to load pages of the site one at a time. The advantages are a much lower memory footprint, faster loading, and much smaller namespaces. This is how php-based programs like WordPress work.

I’ve developed a skeleton for a multi-page, multi-user Shiny web site, with user authentication. You can see/download the code from https://github.com/open-meta/om_skeleton.

Tom


#7

This breakdown is great, and I will definitely be stealing some of your ideas (especially commenting major ui sections and folding them when done with them)!

One thing I will say regarding this:

If you do use a global.R you can just source your "functions.R" file from there. I sourced my external files into both my ui.R and server.R files separately when I first started using shiny and switching to global.R has saved myself from a lot of errors where I changed something that is outside of the ui or server elements but forgot to change it in the other.


#8

This is really helpful, thanks Mark! That html template is a beauty, I’ll definitely be testing it out.

Great advice from everyone on workflow, I’ve already been putting a lot it into practice and am really seeing the benefit in writing modules and UI functions to handle repetitive code blocks.


#9

I’ll do that, I forgot about global.R's functionality actually so good thread :+1:


#10

I have been looking very closely to the radiant collection of UI packages. It looks very well organized with a standard UI all the way for all the packages. radiant creator is Vincent Nijs. https://github.com/radiant-rstats/radiant. All of these packages use the same underlying UI.

This is the folder structure for the radiant_basics package. Additionally, there are: radiant.data, radiant.design, radiant.model, and radiant.multivariate.

Would you think this is a good pattern to follow?

-- R
	|--- aaa.R
	|--- compare_means.R
	|--- compare_props.R
	|--- correlation.R
	|--- cross_tabs.R
	|--- goodness.R
	|--- prob_calc.R
	|--- radiant.R
	|--- single_mean.R
	|--- single_prop.R
--- inst
	|--- app
		|--- global.R
		|--- help.R
		|--- init.R
		|--- server.R
		|--- tools
			|--- analysis
				|--- clt.R
				|--- compare_means_ui.R
				|--- compare_props_ui.R
				|--- correlation_ui.R
				|--- cross_tabs_ui.R
				|--- goodness_ui.R
				|--- prob_calc_ui.R
				|--- single_mean_ui.R
				|--- single_prop_ui.R
				
			|--- help
				|--- clt.md
				|--- compare_means.md
				|--- compare_props.md
				|--- correlation.md
				|--- cross_tabs.md
				|--- figures
					|---
				|--- goodness.md
				|--- prob_calc.md
				|--- single_mean.md
				|--- single_prop.md
			
		|--- ui.R
		|--- www
			|--- js
				|--- run_return.js
			
	|--- rstudio
		|--- addins.dcf


#11

@nutterb Putting a shiny application inside a dedicated package is certainly one approach I am pursuing. Have you deployed these applications on an internal Shiny server or RStudio Connect server? At my organization I’ve always deployed apps using the typical ui.R / server.R or app.R constructs inside the Git repository for the app, which are then pushed to our internal shiny servers. I’m just curious how you would handle a deployment when the app is inside a package.


#12

I’m not a great person to ask about deployment. I work on a project that only allows Windows based servers. This means Shiny Server and RStudio Connect servers are not an option for me, since they only operate on Linux distributions.

In the simple form, I use a .bat file to kick off an R script that initiates the applications, runs it on a designated port, and then people connect to the app using a URL that looks like http://servername:port.

This is a pretty poor deployment solution. It was just the first thing I could come up with that we could maintain with relative ease. If RStudio ever releases Shiny Server that can be run on a Windows machine, we might change the way we do things (hint, hint!)