I have a 50k+ line script, no trace back, no glimpse.
I'm triggering a sliding window function via runner::runner(), which does a lot of stuff... doParallel, anova_racing, nesting etc.
It all works, until I expand the amount of data/windows to a specific point. I have no clue, even lobstr::cst() does not show any direction to go.
What is the most likely reason for this....
R Error: node stack overflow
R Error during wrapup: node stack overflow
Error: no more error handlers available (recursive errors?); invoking 'abort' restart
Looks like it’s set in the OS environment variable. Maybe (?) running it in the terminal console outside RStudio might make it squeeze through.
As far as finding the recursive culprit in 50K lines? Might be possible if there are only a couple of places it’s involved. If I found them, I’d try to save the input argument just before it’s called, and then try to profile just that function.
The group of sysadmins for technical hardware and software support at WU Vienna can be contacted via E-mail: CRAN-sysadmin at R-project.org
Should be feasible to roll your own, especially on Debian. Other possibilities are to save the data state and delegate the recursive bits to a stand alone C/C++ tool. If the R equivalent makes use of shared libraries that might be a simple wrapper. Or could be an occasion to start with Julia, which can import R functions without the R runner. That could be a speedy way to execute the code.
R was built as a lab, not a factory, for comfort, not for speed. As part of a project to be put into production, it’s superb for prototyping, not so much as something fast and robust to OS perturbations. A big handicap however is that R is designed as a functional language as it presents to the user. As an imperative/procedural language, however, it is lame. That results in a drag on porting to compiled languages, the most common of which have to reverse engineer the functional logic or clean up the crude procedural cruft. Not too many shops that don’t face this obstacle except the few using Haskell, for example. Seems to account nicely for the data science popularity of Python.