Read tabular information from a webpage using an R script

tidyverse
1

Hi,

I want to download the stations list from the link below

http://www.sciamachy-validation.org/klimatologie/monv/nstations.html

This contains the list of all Dutch rain data collection stations with there coordinates, names and station ids.

What read function allows me to read this data from the web from an R script.

Thanks a lot.

Fritsander

2

I think you're looking for rvest

https://stat4701.github.io/edav/2015/04/02/rvest_tutorial/

2 Likes
3

It is correct you need to use rvest :package:. I advice to read a more recent example like this one:
Scraping CRAN with rvest to get a look at how it works.
You will find a lot of example on the net by the rstats community.

Here is the few lines to get back the table you want

library(rvest)
#> Le chargement a nécessité le package : xml2

website <- read_html("http://www.sciamachy-validation.org/klimatologie/monv/nstations.html")

tab <- website %>%
  html_nodes(css = "table .onelinetable") %>%
  .[[1]] %>%
  html_table()

str(tab)
#> 'data.frame':    321 obs. of  6 variables:
#>  $ Locatie: chr  "Aalsmeer" "Aalten" "Abcoude" "Akkrum" ...
#>  $ Nr     : int  458 680 572 89 664 678 560 910 835 171 ...
#>  $ OL     : chr  "4° 46'" "6° 34'" "4° 58'" "5° 49'" ...
#>  $ NB     : chr  "52° 15'" "51° 54'" "52° 15'" "53°  3'" ...
#>  $ X (km) : int  113 236 127 184 242 218 160 143 132 204 ...
#>  $ Y (km) : int  475 437 475 563 485 463 445 417 422 600 ...

Created on 2017-12-12 by the reprex package (v0.1.1.9000).

html_nodes works using css or xpath selector. I look into the website code source to know how to select your table. You can use SelectorGadget following rvest CRAN vignette.

I advice on reading some ressources if you want to play further with rvest :package:

1 Like
4

@cderv very nice code, and a good example for using CSS selectors to scrape a table.

If you are looking for further tutorials on using rvest, I would recommend you look at this blog for a good walkthrough of using rvest for Web scraping:
https://medium.com/@peterjgensler/functions-with-r-and-rvest-a-laymens-guide-acda42325a77

2 Likes
5

Thanks a lot! I'll be reading the tutorials you sent me.

Regards,

Fritsander

1 Like