My shiny disconnect from server without any error, works locally fine

shiny
1

Hi,

I have been debugging my shinyapp for a day. The app works find locally. But it keeps showing disconnected from server after I published it. Can anyone help me? I would appreciate it a lot if the problem could be solved. Here is the URL of the app: https://jiaruitang.shinyapps.io/myapp-1/

Here are the application logs:
2018-11-11T23:44:32.431341+00:00 shinyapps[564753]:
2018-11-11T23:44:32.431342+00:00 shinyapps[564753]: intersect, setdiff, setequal, union
2018-11-11T23:44:32.431343+00:00 shinyapps[564753]:
2018-11-11T23:44:32.535717+00:00 shinyapps[564753]:
2018-11-11T23:44:32.535719+00:00 shinyapps[564753]: Attaching package: ‘XML’
2018-11-11T23:44:32.535721+00:00 shinyapps[564753]:
2018-11-11T23:44:32.536090+00:00 shinyapps[564753]: The following object is masked from ‘package:rvest’:
2018-11-11T23:44:32.536092+00:00 shinyapps[564753]: xml
2018-11-11T23:44:32.536091+00:00 shinyapps[564753]:
2018-11-11T23:44:32.536093+00:00 shinyapps[564753]:
2018-11-11T23:44:32.540605+00:00 shinyapps[564753]: Loading required package: bitops
2018-11-11T23:44:32.642317+00:00 shinyapps[564753]: Loading required package: sp
2018-11-11T23:44:32.797685+00:00 shinyapps[564753]: rgdal: version: 1.2-16, (SVN revision 701)
2018-11-11T23:44:32.797691+00:00 shinyapps[564753]: Geospatial Data Abstraction Library extensions to R successfully loaded
2018-11-11T23:44:32.797692+00:00 shinyapps[564753]: Loaded GDAL runtime: GDAL 2.1.3, released 2017/20/01
2018-11-11T23:44:32.797693+00:00 shinyapps[564753]: Path to GDAL shared files: /usr/share/gdal/2.1
2018-11-11T23:44:32.797693+00:00 shinyapps[564753]: GDAL binary built with GEOS: TRUE
2018-11-11T23:44:32.797694+00:00 shinyapps[564753]: Loaded PROJ.4 runtime: Rel. 4.9.2, 08 September 2015, [PJ_VERSION: 492]
2018-11-11T23:44:32.797694+00:00 shinyapps[564753]: Path to PROJ.4 shared files: (autodetected)
2018-11-11T23:44:32.797695+00:00 shinyapps[564753]: Linking to sp version: 1.2-7
2018-11-11T23:44:32.837545+00:00 shinyapps[564753]: [1] "loading of ui: 0.000518798828125"
2018-11-11T23:44:34.843208+00:00 shinyapps[564753]:
2018-11-11T23:44:34.843210+00:00 shinyapps[564753]: Attaching package: ‘hashmap’
2018-11-11T23:44:34.961311+00:00 shinyapps[564753]:
2018-11-11T23:44:34.843211+00:00 shinyapps[564753]:
2018-11-11T23:44:34.961312+00:00 shinyapps[564753]: inset
2018-11-11T23:44:34.843562+00:00 shinyapps[564753]: The following object is masked from ‘package:RCurl’:
2018-11-11T23:44:34.961313+00:00 shinyapps[564753]:
2018-11-11T23:44:34.843563+00:00 shinyapps[564753]:
2018-11-11T23:44:34.848427+00:00 shinyapps[564753]: Loading required package: ggplot2
2018-11-11T23:44:34.960920+00:00 shinyapps[564753]:
2018-11-11T23:44:34.961310+00:00 shinyapps[564753]: The following object is masked from ‘package:magrittr’:
2018-11-11T23:44:34.960916+00:00 shinyapps[564753]:
2018-11-11T23:44:34.843565+00:00 shinyapps[564753]:
2018-11-11T23:44:34.843564+00:00 shinyapps[564753]: clone
2018-11-11T23:44:34.960919+00:00 shinyapps[564753]: Attaching package: ‘ggmap’
2018-11-11T23:44:37.402845+00:00 shinyapps[564753]: [1] "loading of h1b2017.rdata: -2.42400622367859"
2018-11-11T23:44:37.404682+00:00 shinyapps[564753]: [1] "loading of industry.rdata: -0.00167083740234375"
2018-11-11T23:44:37.407351+00:00 shinyapps[564753]: [1] "loading of server: -0.00213527679443359"
2018-11-11T23:44:37.571184+00:00 shinyapps[564753]: OGR data source with driver: ESRI Shapefile
2018-11-11T23:44:37.571186+00:00 shinyapps[564753]: Source: "USA_adm_shp", layer: "USA_adm1"
2018-11-11T23:44:37.571220+00:00 shinyapps[564753]: with 52 features
2018-11-11T23:44:37.571222+00:00 shinyapps[564753]: It has 12 fields
2018-11-11T23:44:37.571245+00:00 shinyapps[564753]: Integer64 fields read as strings: ID_0 ID_1 CCN_1
2018-11-11T23:44:40.111439+00:00 shinyapps[564753]: [1] "shinyserver shape read: 2.70014452934265"
2018-11-11T23:44:40.147797+00:00 shinyapps[564753]: [1] "ipop time: 0.00149059295654297"
2018-11-11T23:44:40.153309+00:00 shinyapps[564753]: [1] "map time: 0.00538492202758789"
2018-11-11T23:44:40.156013+00:00 shinyapps[564753]: [1] "state table time: 0.00256967544555664"
2018-11-11T23:44:40.158531+00:00 shinyapps[564753]: [1] "industry table time: 0.00237607955932617"
2018-11-11T23:45:02.722888+00:00 shinyapps[564753]: [1] "histogram time: 0.401571035385132"
2018-11-11T23:45:10.288538+00:00 shinyapps[564753]: [1] "histogram time: 0.0258960723876953"

Output of "rsconnect::appDependencies()":

package   version source
1            BH  1.66.0-1   CRAN
2            DT       0.4   CRAN
3          MASS    7.3-48   CRAN
4        Matrix    1.2-12   CRAN
5            R6     2.2.2   CRAN
6  RColorBrewer     1.1-2   CRAN
7         RCurl 1.95-4.10   CRAN
8          Rcpp   0.12.15   CRAN
9   RgoogleMaps     1.4.1   CRAN
10          XML  3.98-1.9   CRAN
11   assertthat     0.2.0   CRAN
12    base64enc     0.1-3   CRAN
13        bindr       0.1   CRAN
14     bindrcpp       0.2   CRAN
15       bitops     1.0-6   CRAN
16   cellranger     1.1.0   CRAN
17          cli     1.0.0   CRAN
18   colorspace     1.3-2   CRAN
19       crayon     1.3.4   CRAN
20    crosstalk     1.0.0   CRAN
21         curl       3.1   CRAN
22   data.table  1.10.4-3   CRAN
23    dichromat     2.0-0   CRAN
24       digest    0.6.15   CRAN
25        dplyr     0.7.4   CRAN
26    geosphere     1.5-7   CRAN
27        ggmap     2.6.1   CRAN
28      ggplot2     2.2.1   CRAN
29      ggrepel     0.7.0   CRAN
30         glue     1.2.0   CRAN
31    gridExtra       2.3   CRAN
32       gtable     0.2.0   CRAN
33      hashmap     0.2.2   CRAN
34          hms     0.4.1   CRAN
35    htmltools     0.3.6   CRAN
36  htmlwidgets       1.0   CRAN
37       httpuv     1.3.5   CRAN
38         httr     1.3.1   CRAN
39         jpeg     0.1-8   CRAN
40     jsonlite       1.5   CRAN
41     labeling       0.3   CRAN
42      lattice   0.20-35   CRAN
43     lazyeval     0.2.1   CRAN
44      leaflet     2.0.0   CRAN
45     magrittr       1.5   CRAN
46      mapproj     1.2-5   CRAN
47         maps     3.2.0   CRAN
48     markdown       0.8   CRAN
49         mime       0.5   CRAN
50      munsell     0.4.3   CRAN
51      openssl       1.0   CRAN
52      packrat   0.4.8-1   CRAN
53       pillar     1.1.0   CRAN
54    pkgconfig     2.0.1   CRAN
55        plogr     0.1-1   CRAN
56         plyr     1.8.4   CRAN
57          png     0.1-7   CRAN
58        proto     1.0.0   CRAN
59       raster     2.6-7   CRAN
60        readr     1.1.1   CRAN
61       readxl     1.0.0   CRAN
62      rematch     1.0.1   CRAN
63     reshape2     1.4.3   CRAN
64        rgdal    1.2-16   CRAN
65        rjson    0.2.15   CRAN
66        rlang     0.1.6   CRAN
67        rvest     0.3.2   CRAN
68       scales     0.5.0   CRAN
69      selectr     0.3-2   CRAN
70        shiny     1.0.5   CRAN
71  sourcetools     0.1.6   CRAN
72           sp     1.2-7   CRAN
73      stringi     1.1.6   CRAN
74      stringr     1.2.0   CRAN
75       tibble     1.4.2   CRAN
76         utf8     1.1.3   CRAN
77      viridis     0.5.0   CRAN
78  viridisLite     0.3.0   CRAN
79         xml2     1.2.0   CRAN
80       xtable     1.8-2   CRAN
81         yaml    2.1.16   CRAN

I have upgraded my load to 8GB, but the problem still exists.

Here are the matrices of CPU Usage (I think this might be where the problem sits):

20
201104×343 14.3 KB

Here is the the code of ui.R:

library(stringr)
library(rvest)
library(Matrix)
library(readr)
library(data.table)
library(magrittr)
library(dplyr)
library(XML)
library(RCurl)
library(rgdal)
library(leaflet)
library(maps)

start_time <- Sys.time()
load("try.RData")

load("occupation.RData")
end_time <- Sys.time()
print(paste0("loading of ui: ", end_time-start_time))

## Page
shinyUI(navbarPage("Map",
                 tabPanel("Interactive Map",
                          div(class="outer",
                              tags$style(type = "text/css", ".outer {position: fixed; top: 41px; left: 0; right: 0; bottom: 0; overflow: hidden; padding: 0}"),
                              leafletOutput("promap", height = "100%")),
                          absolutePanel(
                            bottom = -70, right = 10, width = 320,
                            draggable = TRUE,
                            wellPanel(
                              selectInput("state", "State", statename, selected = statename[1]),
                              selectInput("occupation", "Occupation", occupationname),
                              sliderInput("bins", "Number of bins:", min =15, max = 65, value = 40),
                              plotOutput("histSalary", height = 180)
                              #plotOutput("histOccup", height = 180)
                            ),
                            style = "opacity:0.85"
                          )),
                 tabPanel("H1B Info by State",
                          fluidRow(
                            column(3,
                                   selectInput("states", "States", statename, multiple=TRUE)
                            )
                            
                          ),
                          hr(),
                          DT::dataTableOutput("by.state")),
                 tabPanel("H1B Info by Industry",
                          fluidRow(
                            column(3,
                                   selectInput("industries", "Industries", industryname, multiple=TRUE)
                            )
                            
                          ),
                          hr(),
                          DT::dataTableOutput("industry.table")
                 ),
                 tabPanel("H1B Graph by Industry",
                          fluidPage(
                            fluidRow(
                              column(8, 
                                     selectInput("forindustries", "Industries", industryname, multiple=F, selected = industryname[1]),
                                     "Choose the industry you want to learn more. The companies are sorted by their number of applications for H1B."
                              )
                            ),
                            plotOutput("barIndustry")
                          )
                 )
                 
))

Here is the code for server:

library(stringr)
library(rvest)
library(Matrix)
library(readr)
library(data.table)
library(magrittr)
library(dplyr)
library(XML)
library(RCurl)
library(leaflet)
library(rgdal)
library(shiny)
library(maps)
library("readxl")
library("hashmap")
library("ggmap")
library("ggrepel")

start_time <- Sys.time()
## Read h1b2017
load("h1b2017.RData")
end_time <- Sys.time()
print(paste0("loading of h1b2017.rdata: ", start_time-end_time))

start_time <- Sys.time()
## Get industry table
load("industry.RData")
end_time <- Sys.time()
print(paste0("loading of industry.rdata: ", start_time-end_time))

load("try.RData")

## Data tidying process
start_time <- Sys.time()
final = read.csv("final.csv")
colnames(final)[1]="NAME_1"
final1 = final
colnames(final1)[1] = "state"
end_time <- Sys.time()
print(paste0("loading of server: ", start_time-end_time))

shinyServer(function(input, output) {
  start_time <- Sys.time()
  dns <- "USA_adm_shp" 
  shape <- readOGR(dns, "USA_adm1")
  end_time <- Sys.time()
  print(paste0("shinyserver shape read: ", end_time-start_time))
  
  start_time <- Sys.time()
  shape2 <- merge(shape, final, by = "NAME_1")
  end_time <- Sys.time()

  start_time <- Sys.time()
  i_popup <- paste0("<strong>STATE: </strong>", shape2$NAME_1, "<br>","<strong>Number of applicants: </strong>", shape2$numbofLCA,"<br>", "<strong>Large Company Ratio: </strong>", shape2$Large_Company_Ratio,
                    "<br>" ,"<strong>Average salary: </strong>", shape2$Avesalary,"<br>","<strong>GDP: </strong>", shape2$GDP,"<br>",
                    "<strong>Population: </strong>", shape2$Population,"<br>","<strong>Education level: </strong>", shape2$education_level)
  pal <- colorQuantile("Greens", NULL, n =7)
  end_time <- Sys.time()
  print(paste0("ipop time: ", end_time-start_time))
  
  start_time <- Sys.time()
  output$promap = renderLeaflet({leaflet(shape2) %>% addTiles() %>% setView(-85, 40, zoom = 3) %>% addPolygons(fillColor = ~pal(shape2$numbofLCA), fillOpacity = 0.8, color = "#000000", weight = 1, popup = i_popup)})
  end_time <- Sys.time()
  print(paste0("map time: ", end_time-start_time))
  
  start_time <- Sys.time()
  output$by.state = DT::renderDataTable({final1 %>% filter(is.null(input$states)|state %in% input$states)})
  end_time <- Sys.time()
  print(paste0("state table time: ", end_time-start_time))
  
  start_time <- Sys.time()
  output$industry.table = DT::renderDataTable({
    industry %>% filter(is.null(input$industries)|industry %in% input$industries)})
  end_time <- Sys.time()
  print(paste0("industry table time: ", end_time-start_time))
  
  output$histSalary = renderPlot({
    start_time <- Sys.time()
    if (length(input$state)==0)
      return(NULL)
    hist((h1b2017 %>% filter(is.null(input$state)|h1b2017[,15] %in% input$state))[,10], breaks = input$bins, plot = T, 
         main = paste0("Histogram of Salary in ", input$state), col = which(statename==input$state), xlab = "Wages per year",cex.axis = 0.8, cex.lab = 0.7, cex.main = 0.7)
    end_time <- Sys.time()
    print(paste0("histogram time: ", end_time-start_time))
  }
  )

## Output histIndustry
  output$barIndustry = renderPlot({
    start_time <- Sys.time()
    par(mar=c(5,11,4,2)+0.1,mgp=c(11,1,0))
    barplot((rev(industry %>% filter(is.null(input$forindustries)|industry %in% input$forindustries))$`Average Salary`), 
            main = "Average Salary of Top25 Companies with Highest H1B Applicants", col = sample(25), border = NA,
            names.arg = rev((industry %>% filter(is.null(input$forindustries)|industry %in% input$forindustries))$`H1B Visa Sponsor`[1:nrow(industry %>% filter(is.null(input$forindustries)|industry %in% input$forindustries))]),
            cex.names = 0.85, las = 2, horiz = T)
    end_time <- Sys.time()
    print(paste0("histogram time: ", end_time-start_time))
  })
})

Thank you again for your help.

2

You seem to be reading input files in the app. Are those files being published to the server in the root path?

3

I think I did. I published the whole folder containing all the input files, server.R and ui.R. I found online that I can trace shiny server to see where the problem is. But I don't know how to trace it. Do you have any idea?

4

Not sure I can help more without getting under the hood. In my experience, Shiny will disconnect when it runs out of memory. You might want to try create a sample subset of your input data to confirm this is/is not the issue.

5

If a shinyapps.io application runs out of memory, there will be a log message. See the documentation.

If the files are large and they are loaded inline, then the application may appear to not have started within the time limit for starting applications. Try loading the data reactively.