I set up a shiny up for user to access and visualize pre-computed predictions.
The shinyapp has access to ~95'000 files.
I noticed, that starting up the app takes about 3 minutes and consumes considerable CPU locally, although the data is hosted online and accessed via an internet browser. Once all data is loaded, the app runs just fast.
My question is, how do I speed up the start up loading of the shiny app? And how can I decrease CPU usage of website-shinyapp users?
I split the pre-computed data already in a hierarchical directory structure to not have more than ~2000 files per directory.
I figured it out.
The problem was not the data amount that would only get loaded reactively in small units. Instead, the bottleneck was the choices vector/list of ~95000 entries within my selectInput() function.
Apparently startup time increases almost exponentially with the number of choices.
length(choices) == 1 --> app loads instantly
length(choices) == 3000 --> app loads instantly
length(choices) == 30000 --> app loads within 5 seconds (noticeable delay)
length(choices) == 95000 --> app loads within 3 minutes with high CPU usage
Now I just swapped out selectInput() with textInput() for the user defined input