Loading huge files in background


#1

Hi!

I have an app, where the choices of selectinput look like the following:
choices=read.delim(file="https://storage.googleapis.com/gencode_ch_data/mouse/mouse_searchNames.txt",header=FALSE,stringsAsFactors = FALSE, sep="\t")

So I have 3 species mouse, human drosophila. In my app there are three selectinput ui for each specie, and when I run and deploy the app it takes time initially to load the page, because I have 3 files to be loaded from the google cloud as shown above for three different kind of species.

My question will be is there a way to load these files in the background, or in a way that does not cause initial loading latency issue when I run the app? Because what my app does is to load these files first.

This issue is very obvious on IE11.

I highly appreciate your advice!
Thank you
Best Regards,
Karni


#2

I have encountered similar issues in terms of slow app startup and trying to find the best ways to at least get the UI in front of the user, even if key components are still processing. Here are a couple of ideas to pursue:

  • A recent new feature in shiny is async programming, which enables you to launch long-running tasks as separate R processes behind the scenes. I haven't used it fully in my applications yet, but I am going to do some prototyping in the near future.
  • Speaking of future, async programming is built on top of the promises and future packages. Before async programming came to be, I actually used the future package directly to offload some background processing (in my case file-based operations) and it worked extremely well.

You'll definitely want to try this out on a simple application to gain familiarity. Let us know if you decide to pursue either of these approaches!


#3

KB246, I have a very similar question Minimize delay in starting Shiny app

I found this on SO: https://stackoverflow.com/questions/39044169/can-you-put-a-shiny-progress-bar-in-global-r

Carl writes:
Each time someone goes to your URL it initiates a new instance of R, which will have to reload the CSV, so yes the CSV will be reloaded. – Carl Aug 19 '16 at 18:21

Based on Carl's response having the data preloaded in memory and ready to go for everyone who comes along sounds unlikely.

I hope I am proven wrong.


#4

You may be able to speed up your load times by using a different file format. For non-spatial data feather looks impressive.