It's a bit hard to create a MRE for this sort of thing. It happens only after working with the same RMD file for a while, say running a few chunks multiple times. Suddenly, my rsession process is using 13G of ram even though my data objects are nowhere near that--the bighgest maybe 1M. (gc shows less than 1G)
I finally realized that if I use the knitr dropdown to clear the knitr cache, it clears the rsession memory. But if I forget to do that frequenty, then I need to manually kill the rsession, hopefully not lose recent file changes, and re-run the RMD file.
Is there a way I could have a command I use periodically in the file to perform that same operation?
Alternatively, can someone help me understand how I accumulate two orders of magnitude more memory in my rsession than the data structures I have going in my session?
Don't have a full answer, but a couple suggestions:
Are you looking to clear the actual knitr cache or the rsession memory? There's much more detail on how the caching works in the knitr manual:
The paths of cache files are determined by the chunk option cache.path; by default all cache files are created under a directory cache relative to the current working directory, and if the option value contains a directory (e.g. cache.path=’cache/abc-’), cache files will be stored under that directory (automatically created if it does not exist).
The ps package might help you drill down a bit more into what's taking up memory: