Hello! I am trying to run some UMAP analysis on a very big dataset using R 4.2.2. I am trying to run it on a Windows 11 PC that has 144 Gb of Ram, however looks like RAM usage is capped at 40% and the analysis still crashes due to not enough ram being allocated. I have already marked as "High priority" all Rstudio tasks in the task manager. Not entirely sure what might be limiting the use of all available RAM. Any help with this would be greatly appreciated!
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)
What specific package are you using? If the package is a wrapper for another library the RAM limit might not be in R. In theory R doesn't have a RAM limit unless you explicitly set it, it defaults to the total available RAM in the system.
Thank you for your reply!. Indeed, I called the function from another package iMUBAC::runUMAP . However, I have tried other RAM demanding functions from other packages to test this and RAM usage seems capped still.
As I suspected, the package you are using is a wrapper for a Java library so you need to configure the Java virtual machine to use more RAM. For example, this will set the limit to 140GB
options(java.parameters="-Xmx140G")
I think they have a technical reason for capping RAM usage to 40%, although, I'm not familiar enough with Java to know what that reason might be.
Thank you, I am afraid I forgot to mention that I had already done that. But you bring up a very valid point. I will just run the analysis using the original package directly and see how that goes. Thanks a lot for your help, really appreciated!
Have you done that before launching the Java instance? Have in mind that setting the option afterwards has no retroactive effect. You need to set it before the instance is launched.
I have never used that specific package but I have used other wrappers for Java libraries (e. g. h2o) and that setting does take effect.