Hello RStudio Community,
I'm after some general advice on how to approach a data caching problem. I have a large SQL query which takes approx. 90 seconds to run, the results of which are consumed in a Shiny app which will--eventually--be deployed via shinyapps.io.
The database I'm pinging is updated daily during the early hours of the morning, before everyone gets into work. Due to our shinyapps deployment, my understanding is that it wouldn't be possible to schedule a cron job on the server to run at, say, 6am to save a .Rdata file to Amazon S3 (or equivalent) for faster data loads.
I'm therefore envisioning a solution whereby the first user of the application on a given day creates a 'day cache' for all subsequent users of the application. The first user would have to wait the full 90 seconds to grab the data, the data are cached to S3 and then all subsequent users of the app on that day benefit from much faster application load times.
After some research, I've found several caching packages which might help, including memoise
and cachem
. While I understand the concept of caching and read Winston Chang's excellent post here, I'm not sure it fully gets at my requirement for a daily refresh of the cache. My understanding is perhaps a little wonky but, as my data-grab function doesn't change--the SQL and function calls remain the same--I'm not sure how the app would 'know' that the cache needs to change?
Any help and advice warmly received!
Cheers!