On MacOSX using the rstream package for multiple random number streams reports an error from within RStudio (Error: C stack usage XXXXXXX is too close to the limit), but not from the terminal or the R.app.
The following code fails in RStudio but works from the terminal or the R.app
#install.packages("rstream")
library(rstream)
# Instantiate a new stream (should be able to generate a very large number
# of such streams)
s <- new("rstream.mrg32k3a")
#> Error: C stack usage 7969260 is too close to the limit
Cstack_info()
# size current direction eval_depth
# 7969177 13168 1 2
The Cstack_info() from the terminal (which does not return an error) is analogous.
Cstack_info()
# size current direction eval_depth
# 7969177 10048 1 2
There's a long thread concluding that having rjava in namespace may be a cause. Although rstream doesn't seem to bring it in, it might help to check sessionInfo()