Crashing RStudio Cloud with spatial data...

I'm teaching a class using RStudio Cloud, and it's been going well so far. But today, we were doing some stuff with spatial data (a similar set of code is available here: https://www.amelia.mn/sds236/MakingMaps.html) and when we got to the leaflet piece, everyone's R sessions crashed, losing everything in their Environment and their History :scream:

I think everyone was working in my STAT360 workspace (https://rstudio.cloud/spaces/9069/projects) and we got a variety of error messages. One interesting/weird feature is that after the crash, the sf library could no longer be loaded into most of their new sessions. Here's an error message:

> library(sf)
Error: package or namespace load failed for ‘sf’:
.onLoad failed in loadNamespace() for 'sf', details:
 call: get(genname, envir = envir)
 error: object 'group_map' not found

Sorry if I've crashed Cloud somehow!

group_map is a dplyr function, which I think has been recently updated, as has sf. Also, sf has to be able to link to GEOS 3.6.1, GDAL 2.1.3, PROJ 4.9.3. I can't log into the cloud account, of course, so it's hard to tell what's loaded. Could you post

sessionInfo()

to take a look at the set-up?

What makes me think I did it is that the code was working for me earlier when it was just me, but once I let my students loose we started having problems. Here's the session info:

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lubridate_1.7.4

loaded via a namespace (and not attached):
[1] compiler_3.5.2 magrittr_1.5   tools_3.5.2    Rcpp_1.0.0    
[5] stringi_1.2.4  knitr_1.21     stringr_1.3.1  xfun_0.4      

Ah, that was after the crash. Here is is after loading dplyr, rgdal, leaflet and trying to load sf but failing.

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] leaflet_2.0.2   rgdal_1.4-3     sp_1.3-1        dplyr_0.7.8    
[5] lubridate_1.7.4

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0       later_0.8.0      pillar_1.3.1     compiler_3.5.2  
 [5] bindr_0.1.1      class_7.3-14     tools_3.5.2      digest_0.6.18   
 [9] tibble_2.0.1     lattice_0.20-38  pkgconfig_2.0.2  rlang_0.3.1     
[13] shiny_1.3.0      DBI_1.0.0        rstudioapi_0.9.0 crosstalk_1.0.0 
[17] xfun_0.4         bindrcpp_0.2.2   e1071_1.7-1      stringr_1.3.1   
[21] knitr_1.21       htmlwidgets_1.3  classInt_0.3-1   grid_3.5.2      
[25] tidyselect_0.2.5 glue_1.3.0       R6_2.3.0         purrr_0.2.5     
[29] magrittr_1.5     promises_1.0.1   htmltools_0.3.6  units_0.6-2     
[33] assertthat_0.2.0 xtable_1.8-3     mime_0.6         httpuv_1.5.1    
[37] stringi_1.2.4    crayon_1.3.4    

I can't reproduce the problem, of course, without the code, but I've been running sf without difficulty. Here's what I have after loading the same packages:

sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rgdal_1.4-3   sp_1.3-1      leaflet_2.0.2 sf_0.7-3      dplyr_0.8.0.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1       rstudioapi_0.10  magrittr_1.5     units_0.6-2     
 [5] tidyselect_0.2.5 lattice_0.20-38  xtable_1.8-3     R6_2.4.0        
 [9] rlang_0.3.4      tools_3.5.3      grid_3.5.3       packrat_0.5.0   
[13] e1071_1.7-1      DBI_1.0.0        htmltools_0.3.6  crosstalk_1.0.0 
[17] class_7.3-15     assertthat_0.2.1 digest_0.6.18    tibble_2.1.1    
[21] crayon_1.3.4     shiny_1.3.0      later_0.8.0      purrr_0.3.2     
[25] htmlwidgets_1.3  promises_1.0.1   mime_0.6         glue_1.3.1      
[29] compiler_3.5.3   pillar_1.3.1     classInt_0.3-1   httpuv_1.5.1    
[33] pkgconfig_2.0.2 
> 

What would worry me most is Ubuntu, R, dplyr, Rcpp, rstudioapi, compiler ... just say everything that's not current. I'm not a cloud user, so I don't know whether you, as a user, can do all of that.

I suspect it has to do with the size of the SpatialPolygonsDataFrame we were creating, which is pretty large. What I don't understand is why crashing R then makes sf not work. I'll try to create a more reproducible issue!

1 Like

Have you tried upgrading to the current CRAN or dev version of dplyr (0.8)? I remember having some weirdness like you’re describing after upgrading sf while having an older version of dplyr installed. As @technocrat mentioned, the reference to dplyr::group_map() in the error message makes me think it’s related to this issue.

Hi Amelia,

Sorry you encountered issues using cloud. I think the R session crashes are memory related. I looked up a couple recent projects in your space and it does look like a number of them hit the 1GB memory limit earlier in the day. This would result in a session crash similar to what you describe.

Sean

1 Like

I woke up thinking the same thing. With a hard limit like that, it makes sense that it would shut down the largest packages first, which might be sf.`

Thanks! I'll try to come up with smaller activities for my students in the future :joy:

You should also talk to the RStudio folk about an academic license with more generous limits; I know they have academic rates on their commercial products, so maybe ...

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.