When I'm in your particular situation I use the following solution
-
Commit the cached .rds files to git, but save them in a directory called tmp_data/. The implication being that you should be able to delete everything in that directory and not worry.
- I have a standardised way of naming the
.rds files, e.g. notebook1-object1.rds. So this file contains object1 from notebook1.
-
Each notebook starts with a load_objects() command, something like
load_objects = function(use_cache = FALSE) {
if(!use_cache) invisible(NULL)
pattern = "^notebook1-(.*)\\.rds$"
fnames = list.files(path = "tmp_data/", pattern = pattern, full.names = TRUE)
var = map(fnames, readRDS)
names(var) = str_match(f, pattern)[,2]
list2env(var, envir = .GlobalEnv)
invisible(NULL)
}
- I also have a function for saving intermediate objects
save_rds = function(obj, notebook) {
obj_name = deparse(substitute(obj)) # Gets the object name
fname = glue::glue("tmp_data/{notebook}-{obj_name}.rds")
saveRDS(obj, fname)
}
I find that this is nice compromise between something really complicated, and something efficient.