How To Limit the Amount of RAM RServer/RSession Uses on Windows

I need to limit the amount of RAM RServer/RSession uses on our Windows 2012 server.

I am a system admin who has two users running R Studio on our Server. Each R Studio spawns one or more RSession.exe processes. Sometimes one of the RSession.exe processes takes up too much RAM. Yesterday, it grew to about 60GB, forcing almost everything else running to a halt. (We have 72 GB available, but 48 GB of that is allocated to SQL Server.) I need to make sure that using R doesn’t bring down my server or affect other applications running on it.

Info:

   sessionInfo()
   R version 3.4.3 (2017-11-30)
   Platform: x86_64-w64-mingw32/x64 (64-bit)
   Running under: Windows Server >= 2012 x64 (build 9200)

As suggested in some forums, I tried adding --max-mem-size=4000M to the shortcut they use to start R. This works with RGUI.exe, but doesn’t work with RStudio.exe. When I run memory.limit() from RGui, it returns 4000, as expected. When I run memory.limit() from RStudio, it returns 71999, regardless of the start parameters. And, in any case, I’m not certain this is the right place to set limits. Would a command line switch like this for RStudio affect RSession processes it spawns?

I also tried running memory.limit(4000) from within RStudio, but got the following message: “Warning message: In memory.limit(4000) : cannot decrease memory limit: ignored”

I’ve also read something about setting memory limits in user profiles, but all of the instructions for doing so are Unix oriented and refer to files and directories that don’t exist on our Windows server. (Sorry, I’m not an R person, just a poor sysadmin trying to keep his server alive…)

One post noted that “Just to let you know, support.rstudio.com on 2014/06/10: "We've got it on our list of things to investigate and hope to have a solution soon" (https://stackoverflow.com/questions/23950132/how-to-set-memory-limit-in-rstudio-desktop-version). Is this a bug? If so, what work-arounds are there? If not, how can I limit RSession memory usage so it doesn’t bring down my server?

Thank You,
Robbie