unable to scrape rstudio::conf agenda?

#1

Hi all,

So I thought I would teach myself a little bit about web scraping today and chose the rstudioconf agenda as my target. But it seems to be the case that the company hosting the event database (cvent) has architected their application soas to render scraping impossible, or at least very difficult without tricks I do not know.

Does anyone know if these data are available elsewhere or do you know a trick for scraping cvent content?

Sort of ironic data for a data science conference are not readily available! LOL

Thanks

Rob

2 Likes
#2

We could crowd source it into a google sheet. If we had 3-5 people it wouldn't take too long.

#3

Yes. All that was easy to get was the session type, time, and title. It does seem designed to make scraping difficult.

#4

I copied the raw data into a google sheet. I'm planning on making it pretty this weekend but here's a link to the raw data if it will help anyone.

Conference Agenda Data Raw

#5

I've scraped this data a couple of days ago.
The code is available in the R folder of this repo: https://github.com/mitchelloharawild/rstudioconf20-helper
I have also made a shiny app (function over form) for searching and adding sessions to a Google Calendar: https://shiny.mitchelloharawild.com/rstudioconf20

4 Likes
#6

I wanted to be able to see which sessions are happening concurrently. I'll finish up day 2 over the weekend. I will also try to adjust the table width, but hopefully this is useful.

Agenda By Session

2 Likes
#7

For anyone interested I have added the breakout sessions for day 2. Link is the same as my previous post.

#8

Thanks, the functionality works - and is correct for my timezone!

#9

Your view (showing concurrent sessions), with Mitchell's functionality (show/hide abstract; add to calendar) would be the sweet spot for a schedule. Ideally a link to the live feed or recorded video would be perfect, with share buttons to send interesting talks to colleagues.

The app does a pretty good job of most of these functions, though some people probably aren't aware that anyone can use the app, even if not a paid attendee? I found it almost by accident - search rstudio in your app store.
EDIT The app can also be accessed by browser - [https://event.crowdcompass.com/rstudioconf2020](http://app link to view in browser)

Ideally I guess the agenda website which lacks many of these functions, could (should?) give you a hint that there are alternative methods?

#10

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.