I am trying to create a simple shiny app that allows my coworkers to select and generate monthly reports. I have built the basic functionality of the app and have it running on with shiny-server on Ubuntu 20.04.2. These reports require six large tables all 1M+ rows that I am currently reading in from a zipped folder in the projects directory.
The problem is a these are static files so for example the data is only current up to 2021-10-01. To prevent manually having to download massive tables from the database every month my solution is to pull just the data from BigQuery in the date range from the end of the current tables to the present day and merge it with the existing tables. So instead of downloading 1M+ row tables from BigQuery I'm only getting 20,000 rows.
The issue I'm running into is I don't know how to keep these master tables persistent all across all sessions. I need a user to be able to hit the 'Update Tables' button have that data merge with my master tables and then stay updated whenever the done. For example if all 6 of my tables currently have data stopping at 2021-10-01 and somebody wants to run reports for the month of October they would click the updated button a function would pull data from Bigquery from 2021-10-01 to 2021-11-04(Current Day) and merge that into the existing table they could then run reports for November and go about there day. If somebody different needs a different report they should be able to go to the site see that the tables are currently up to date through 2021-11-04 and run there reports.
I'm currently loading the tables in from a global.R file that looks like this
customers_shiny <<- read.csv(unz("csvdata.zip", "csvdata/customers.csv"), header = T)
invoices_shiny <<- read.csv(unz("csvdata.zip", "csvdata/invoices.csv"), header = T)
charges_shiny <<- read.csv(unz("csvdata.zip", "csvdata/charges.csv"), header = T)
subscriptions_shiny <<- read.csv(unz("csvdata.zip", "csvdata/subscriptions.csv"), header = T)
feedback_shiny <<- read.csv(unz("csvdata.zip", "csvdata/feedback.csv"), header = T)
tracking_shiny <<- read.csv(unz("csvdata.zip", "csvdata/tracking.csv"), header = T)