R Shiny App works great...until I deploy...then 'out of memory' message

[dfhkvs]

Hi. I have a simple app. All it does is pull a small (1-2MB) image from a public google cloud storage bucket and display it on a simple map. Works great on its own. When I deploy, I get an out of memory error. The app is set to 'large' in the settings in shinyapps.io.

Here is the code:

# test the display of a GCS image.

library(shiny)
library(raster)
library(leaflet)

ui = fluidPage(
  
  titlePanel("MountainSnow Map"),
  helpText("Attempt to plot a raster"),
  fluidRow(
    column(12,
           leafletOutput(outputId = 'map'),
           fluidRow(
             column(12,
                    verbatimTextOutput("info")),
             fluidRow(
               column(12,
                      plotOutput("distPlot")),
                    )
                    )
        )
        )
)

server <- function(input, output) {
  
  tmp.url <- "/vsicurl/https://storage.googleapis.com/cso_test_upload/co_n_domain/swed_wi_assim/2023_04_01_swed_wi_assim.tif"
  ras <- raster(tmp.url)
  
  #ras <- writePNG(ras)
  
  #cont. colormap option
  #pal <- colorNumeric(c("#0C2C84", "#41B6C4", "#FFFFCC"), values(ras),
  #                    na.color = "transparent")
  #discrete colormap option
  pal <- colorBin(c("#0C2C84", "#41B6C4", "#FFFFCC"), values(ras), 6, pretty = TRUE,
                      na.color = "transparent")
  
  output$map = renderLeaflet({
    leaflet() %>%
      addTiles() %>%
      addRasterImage(ras, colors = pal, opacity = 0.8) %>%
      setView(lng = -110, lat = 40, zoom = 4) %>%
      addLegend(pal = pal, values = values(ras), title = "SWE (m)")
  })
  
}

# Run the application 
shinyApp(ui = ui, server = server)

And, here are the logs:

2023-04-07T20:01:31.281552+00:00 shinyapps[8793660]: Running on host: e870498d39ed
2023-04-07T20:01:31.283911+00:00 shinyapps[8793660]: Running as user: uid=10001(shiny) gid=10001(shiny) groups=10001(shiny)
2023-04-07T20:01:31.283931+00:00 shinyapps[8793660]: Connect version: 2023.03.0
2023-04-07T20:01:31.283934+00:00 shinyapps[8793660]: LANG: C.UTF-8
2023-04-07T20:01:31.283937+00:00 shinyapps[8793660]: Working directory: /srv/connect/apps/mtnsnow
2023-04-07T20:01:31.284120+00:00 shinyapps[8793660]: Using R 4.2.2
2023-04-07T20:01:31.284130+00:00 shinyapps[8793660]: R.home(): /opt/R/4.2.2/lib/R
2023-04-07T20:01:31.284359+00:00 shinyapps[8793660]: Content will use current R environment
2023-04-07T20:01:31.284365+00:00 shinyapps[8793660]: R_LIBS: (unset)
2023-04-07T20:01:31.284394+00:00 shinyapps[8793660]: .libPaths(): /opt/R/4.2.2/lib/R/library
2023-04-07T20:01:31.291511+00:00 shinyapps[8793660]: shiny version: 1.7.4
2023-04-07T20:01:31.291526+00:00 shinyapps[8793660]: httpuv version: 1.6.9
2023-04-07T20:01:31.291532+00:00 shinyapps[8793660]: rmarkdown version: 2.21
2023-04-07T20:01:31.291536+00:00 shinyapps[8793660]: knitr version: 1.42
2023-04-07T20:01:31.291552+00:00 shinyapps[8793660]: jsonlite version: 1.8.4
2023-04-07T20:01:31.291555+00:00 shinyapps[8793660]: RJSONIO version: (none)
2023-04-07T20:01:31.291559+00:00 shinyapps[8793660]: htmltools version: 0.5.5
2023-04-07T20:01:31.291630+00:00 shinyapps[8793660]: reticulate version: (none)
2023-04-07T20:01:31.291875+00:00 shinyapps[8793660]: Using pandoc: /opt/connect/ext/pandoc/2.16
2023-04-07T20:01:31.716573+00:00 shinyapps[8793660]: Starting R with process ID: '32'
2023-04-07T20:01:31.716962+00:00 shinyapps[8793660]: Shiny application starting ...
2023-04-07T20:01:31.737705+00:00 shinyapps[8793660]: Loading required package: sp
2023-04-07T20:01:37.687147+00:00 shinyapps[8793660]: Listening on http://127.0.0.1:37977
2023-04-07T20:01:47.252315+00:00 shinyapps[8793660]: Container event from container-7981853: oom (out of memory)
2023-04-07T20:08:16.756786+00:00 shinyapps[8793660]: Container event from container-7981872: start
2023-04-07T20:08:17.130165+00:00 shinyapps[8793660]: Running on host: 0957fa82c475
2023-04-07T20:08:17.142536+00:00 shinyapps[8793660]: Running as user: uid=10001(shiny) gid=10001(shiny) groups=10001(shiny)
2023-04-07T20:08:17.142564+00:00 shinyapps[8793660]: Connect version: 2023.03.0
2023-04-07T20:08:17.142567+00:00 shinyapps[8793660]: LANG: C.UTF-8
2023-04-07T20:08:17.142570+00:00 shinyapps[8793660]: Working directory: /srv/connect/apps/mtnsnow
2023-04-07T20:08:17.142828+00:00 shinyapps[8793660]: Using R 4.2.2
2023-04-07T20:08:17.142845+00:00 shinyapps[8793660]: R.home(): /opt/R/4.2.2/lib/R
2023-04-07T20:08:17.143090+00:00 shinyapps[8793660]: Content will use current R environment
2023-04-07T20:08:17.143097+00:00 shinyapps[8793660]: R_LIBS: (unset)
2023-04-07T20:08:17.143105+00:00 shinyapps[8793660]: .libPaths(): /opt/R/4.2.2/lib/R/library
2023-04-07T20:08:17.149581+00:00 shinyapps[8793660]: shiny version: 1.7.4
2023-04-07T20:08:17.149594+00:00 shinyapps[8793660]: httpuv version: 1.6.9
2023-04-07T20:08:17.149597+00:00 shinyapps[8793660]: rmarkdown version: 2.21
2023-04-07T20:08:17.149601+00:00 shinyapps[8793660]: knitr version: 1.42
2023-04-07T20:08:17.149607+00:00 shinyapps[8793660]: jsonlite version: 1.8.4
2023-04-07T20:08:17.149630+00:00 shinyapps[8793660]: RJSONIO version: (none)
2023-04-07T20:08:17.149643+00:00 shinyapps[8793660]: htmltools version: 0.5.5
2023-04-07T20:08:17.149657+00:00 shinyapps[8793660]: reticulate version: (none)
2023-04-07T20:08:17.149964+00:00 shinyapps[8793660]: Using pandoc: /opt/connect/ext/pandoc/2.16
2023-04-07T20:08:17.536378+00:00 shinyapps[8793660]: Starting R with process ID: '32'
2023-04-07T20:08:17.536772+00:00 shinyapps[8793660]: Shiny application starting ...
2023-04-07T20:08:17.557956+00:00 shinyapps[8793660]: Loading required package: sp
2023-04-07T20:08:19.185224+00:00 shinyapps[8793660]: Container event from container-7981853: stop
2023-04-07T20:08:22.891481+00:00 shinyapps[8793660]: Listening on http://127.0.0.1:34623
2023-04-07T20:08:32.219967+00:00 shinyapps[8793660]: Container event from container-7981872: oom (out of memory)

[smruthi]

Hi,
Could you resolve it? Facing the same issue and lookin for work arounds.
thanks

[AlexisW]

When opening your administration page, in Settings > General, you can select the instance size. Make sure you use the biggest size available (I think free accounts are limited to "Large" with 1 GB).

When running on your computer, you can try monitoring memory to see how much is needed, that might help you see if upgrading to a slightly more expensive shinyapps.io tier could solve the problem.

In the end, you might have to determine which part of the code is using up the memory; in the original I guess it could be the maps (I don't know much about maps, I could be wrong), with other type of data there might be ways to use a database as a more efficient storage.

[dfhkvs]

No solution yet, smruthi.

[dfhkvs]

Alexis.
Thank you. I have upgraded to a higher tier, but this still did not solve it. The raster image that I am trying to display is modest in size (1-10MB). So the 'out of memory' message I get in the logs puzzles me still.

[smruthi]

I reduced the datasets. I retained only the necessary columns and removed additional columns.
It works for me without an upgrade.

[AlexisW]

It's difficult to give a general answer, as that really depends on what you're doing. Even if the final raster image is small, maybe some intermediate steps are using up a lot of memory.

When running this code on your computer, can you see how much memory it uses? You can try using {profvis}, which gives you some estimates of memory, or just use your OS's task manager/resource monitor to look at what's happening. Both of these methods are not fully accurate, but give you an idea.

When I run your reprex code on my computer with profvis (not in a Shiny app), it looks like it's using 4.5 GB (but freeing up 3.5 GB), of these 3 GB in the call to addRasterImage(). So, following this, I tried adding project = FALSE, and indeed it strongly reduced the amount of memory needed (to 15 MB). This is also discussed here. Maybe that would help in your case?