RStudio Server - UI Script Running vs. Local Jobs Script Performance (and Environments)

I am observing significant performance differences and memory consumption between running a script live as compared to submitting it as a local job to the community edition of RStudio Server.

Background:

  • Running Ubuntu 18.04 LTS with RStudio Server (Version 1.2.1335). Server has 64GB of RAM and 16 cores.
  • I have a long running optimization script using the library(GA) (genetic algorithms) library which makes use of the library(doParallel) to provide parallelization across the cores.

Observations:

  • When I run the script via the console or user interface, the process generally takes 1.25 hours and consumes 20% of the 64GB of RAM for the duration. Average processor utilization is 80% of the 16 cores for the duration.
  • When I submit the script as a job I am observing different behavior. The memory usage spikes to nearly 100% and then encroaches on swap. Following that, processor utilization stays around 1-2% and then every 5 minutes jumps to about 50% for a few seconds. I let the script run for about 2 hours before stopping.

Other Facts:

  • Both scenarios above have no other user initiated processes running on the server or during the job submission.
  • The script is self-contained, sourcing other required R scripts as well as RDS files as needed from the working directory.

Research to Date:

  • I've reviewed the RStudio Server Pro manual and found the rserver.conf and rsession.conf options. They are both currently blank.
  • I have a theory the inconsistent behavior may either have to do with:
    • The R environment and 'copy job results values', however have not found a resolution yet
    • The GA library loading in the job created environment and recognizing the available server resources available (i.e. # of cores). This would not explain the high memory usage though.

Any help or references to overlooked research areas is greatly appreciated. Thank you!