Multiple NUMA Nodes in R?

I have a server with 224 logical cores across 4 NUMA nodes. I am trying to thread an existing R script that works on 1 NUMA node (56 cores) using parallel and foreach. Python detects all 224 cores, but detectCores only detects 56 cores on one NUMA node. Is there an R setting to use all NODEs?

Have a look at this thread from r-devel:
In particular this summary message:

1 Like

That will detect the cores, but does not mean that I can use them. Is there a fix for making a parallel cluster?

[1] 224
Error in file(con, "w") : all connections are in use

I assume that is a HyperV problem? There are 112 cores, but 224 logical processors.


I think you hit a hard limit in R, c.f. connections.c and ?connections

So currently there is a maximum of 125 connections... Whats the process of requesting this to be updated? I would rather not rewrite this application in Python...

You could try compiling R from source with an increased value. To get this changed officially, you should ask on r-devel.

1 Like

It is hard coded in the connections.c file of R. I will update that value to 224 and update later today. Thanks for your help.

#ifdef Win32
# include <Startup.h>

#define NCONNECTIONS 128 /* snow needs one per slave node */
#define NSINKS 21