"Upload Complete" (fileInput) takes a long time in Shiny

shiny
#1

It took 20-25 seconds to see "Upload Complete" when I upload a 30 MB csv file in Shiny.

The upload bar completes almost instantaneously. The "Upload Complete" takes another 20 seconds to show up.

According to https://stackoverflow.com/questions/54586226/r-shiny-fileinput-displays-upload-completed-a-few-seconds-before-it-is-actual, the upload bar measures the time to upload the file into the temp directory. Not the time to read it into memory.

Thus, it looks like it takes 20 seconds for it to read my data into memory. Any idea how to speed this up?

My code is very simple:

app.R

library(shiny)

options(shiny.maxRequestSize = 50*1024^2)

server <- function(input, output, session) {
  rawdata <- shiny::eventReactive (input$inFile, {
    rdata <- capture.output(data.table::fread(input$inFile$datapath, header=input$header, sep=",", data.table = F, verbose = T))
  })
}

ui <- fluidPage(
  titlePanel("title panel"),
  
  sidebarLayout(
    sidebarPanel("sidebar panel",
                 shiny::fileInput(inputId = "inFile", "Choose a CSV File",
                                  accept = c(
                                    "text/csv",
                                    "text/comma-separated-values,text/plain",
                                    ".csv"
                                  )
                 ),
                 
                 shiny::checkboxInput("header", "Header", TRUE)),
    mainPanel("main panel")
  )
)

shinyApp(ui,server)
0 Likes

#2

What is your motivation for using capture.output? That can be a big pain point with speed.

Trying using it directly with verbose = F

rdata <- data.table::fread(
  input$inFile$datapath, 
  header = input$header, 
  sep = ",", 
  data.table = F, 
  verbose = F
)
0 Likes

#3

I used capture.out just to see the results of verbose = T. The process still takes 20 seconds with verbose = T or verbose = F.

0 Likes

#4

I'd look at this app / uploading slowing while using profvis. https://rstudio.github.io/profvis/examples.html#example-3---profiling-a-shiny-application

This should provide some insight to the slowdown. (Please post a screen shot of your profvis output.)

1 Like

fileInput is slow in Shiny
#5

Here is the output. Let me know if you see anything that explains the issue. Thank you.

0 Likes

#6

Hi, thank you for the report. This is definitely weird, but I wasn't able to reproduce it myself on my machine with this 60M CSV. When I ran your app and uploaded the file I saw "Upload Complete" after about 4 seconds.

Is this app running locally from the IDE on your laptop, or remotely on a server somewhere?

Do you have any evidence that the particular CSV takes a long time to parse for any reason? You could test this by running just data.table::fread("~/path/to/your.csv", header=T, sep=",", data.table = F) and seeing if it was slow.

1 Like

#7

Hi Alan, thanks for your reply.

I am running it locally from the IDE on my laptop.

No, I don't have any evidence this particular CSV takes a long time. When I upload your 60M CSV file using the app, it takes me almost 1 minute.

When I run just data.table::fread("~/path/to/your.csv", header=T, sep=",", data.table = F), the data is uploaded almost instantly.

What else can I check? FYI, I am using RStudio Version 1.2.1335, R version 3.6.0 (2019-04-26), shiny_1.3.2

0 Likes