read.csv for web based csv file works locally, but not when hosted on shinyapps.io

Posit Professional & Hosted Products shinyapps.io
1

This code works fine when run locally (Windows 10), but gives me the following error when I try to run it through shinyapps[dot]io:

Warning in open.connection(file, "rt") : cannot open URL 'https://www.thepredictiontracker.com/ncaapredictions.csv': HTTP status was '403 Forbidden'

The little bit that I could find online suggested that there may be some issues with shinyapps[dot]io's Linux environment handling URLs, but I can't figure out how to correct for this.

Does anyone have a workaround?

ui <- fluidPage(
fluidRow(
column(12, dataTableOutput("ncaaf_table"))
)
)

server <- function(input, output) {
  output$ncaaf_table <- renderDataTable({
    read.csv(url("https://www.thepredictiontracker.com/ncaapredictions.csv"))
  })
}
2

shinyapps.io uses Linux so you may want to switch from using url to something like httr::GET.

Can you try this instead? httr::GET("https://www.thepredictiontracker.com/ncaapredictions.csv")

1 Like
3

The HTTP 403 error means that the server (at thepredictiontracker.com) refused the connection from shinapps.io

Does access to the csv you want on thepredictiontracker.com have any authentication requirements (that you maybe have configured locally but which wouldn't existing on shinyapps)?

4

As far as I know there isn't any authentication required for that website, but that would make sense. Maybe it has some kind of protections built in to prevent abuse from scraping bots or something.

5

This also works great locally, but then fails when pushed to the shinyapps.io environment.

6

I had the same issue, here is how I fixed it:

n <- getURL("https://www.urlforsite.com/data/data.tsv", ssl.verifypeer = FALSE)
newData <- read.csv(textConnection(n), stringsAsFactors = F,sep = "\t",header = F)

While setting the SSL to not verify is obviously not an optimal solution, it enabled me to get the application back online.

1 Like