I'm new in web scraping using R. I'm trying to scrape the table generated by this link: https://gd.eppo.int/search?k=saperda+tridentata. In this specific case, it's just one record in the table but it could be more (I am actually interested in the first column but the whole table is ok).
I already looked for this issue somewhere else and I tried to apply the tips found here but with no success (maybe for my little knowledge on how web pages work): https://stackoverflow.com/questions/59312399/rvest-table-with-thead-and-tbody-tags. Maybe I am not following correctly the suggested step "# Get the JSON as plain text from the link generated by Javascript on the page". Where can I get this link? In this specific case I used the json link reported in the script below... is that the correct one in the case of the page I am exploring?
I already explored the option RSelenium... but I had so many problems that I quit.
In my work laptop there might be problems related to corporate limitations (I am not administrator of my machine). So I cannot even establish a connection. Error message: "Undefined error in httr call. httr output: Failed to connect to localhost port 444: Connection refused". The same with port 4567.
Using my personal pc I copied and pasted your script. I tried using both Chrome and Firefox, but I always get an empty table.... this is really weird.... I have no idea why this is happening. The result of this is an empty table again (0 obs. of 5 variables). I am wondering if this is happening for an html issue. Not sure I am reading correctly the html nodes. But I am not at all expert on html so I cannot even explain what I am trying to say
no suggestions for 1. If you get it to work on personal device, you could ask administrator for advice.
the second time I run the script it get errors with the message that the port is already in use (even when I close with
remDr$close()
# stop the selenium server
rD[["server"]]$stop()
rm(rD)
gc(verbose=F)
I have to close RStudio before proceeding or use another port number. But you don't get errors, only a empty table (?) Could you include the relevant portion of the code that you use? Or is it exactly the same as the code that I used?
I am having the same port issue: I am not able to close it.
When the port is working, I use exactly the same code that you use... and this is why it's weird that you can get the data and I can't...
I am afraid that there is some issue (in my machine) with the correct interpretation of the html tag tbody which determines where the data in a html table are (as far as I understood...)
Can you please try to use the following code and tell me if you can get the table?