HORROR! 50k+ line script -> node stack overflow

I have a 50k+ line script, no trace back, no glimpse.

I'm triggering a sliding window function via runner::runner(), which does a lot of stuff... doParallel, anova_racing, nesting etc.
It all works, until I expand the amount of data/windows to a specific point. I have no clue, even lobstr::cst() does not show any direction to go.

What is the most likely reason for this....

R Error: node stack overflow
R Error during wrapup: node stack overflow
Error: no more error handlers available (recursive errors?); invoking 'abort' restart

See the first answer on the S/O post.

(Good grief; it's an interpreted language!)

And in the mean time, is there a command line option for it?
How can we identify the offending recursion?

Looks like it’s set in the OS environment variable. Maybe (?) running it in the terminal console outside RStudio might make it squeeze through.

As far as finding the recursive culprit in 50K lines? Might be possible if there are only a couple of places it’s involved. If I found them, I’d try to save the input argument just before it’s called, and then try to profile just that function.

As far as I understand, we have to compile our own R to get more tolerance on stack size.

The default limits could be more then 5 years old. Meanwhile we have extensive mappings, nestings, stronger hardware and more data in general.

I' m wondering who is responsible to maintain this numbers.

#define R_BCNODESTACKSIZE 10000
#define R_BCINTSTACKSIZE 10000

I think that group is

The group of sysadmins for technical hardware and software support at WU Vienna can be contacted via E-mail: CRAN-sysadmin at R-project.org

Should be feasible to roll your own, especially on Debian. Other possibilities are to save the data state and delegate the recursive bits to a stand alone C/C++ tool. If the R equivalent makes use of shared libraries that might be a simple wrapper. Or could be an occasion to start with Julia, which can import R functions without the R runner. That could be a speedy way to execute the code.

R was built as a lab, not a factory, for comfort, not for speed. As part of a project to be put into production, it’s superb for prototyping, not so much as something fast and robust to OS perturbations. A big handicap however is that R is designed as a functional language as it presents to the user. As an imperative/procedural language, however, it is lame. That results in a drag on porting to compiled languages, the most common of which have to reverse engineer the functional logic or clean up the crude procedural cruft. Not too many shops that don’t face this obstacle except the few using Haskell, for example. Seems to account nicely for the data science popularity of Python.

The response of CRAN-sysadmin@R-project.org is:

Please ask on the R-devel mailing list, this mailig list is for the
technical maintainance of the CRAN servers.