Trying to scrape job descriptions on Indeed.com

[mara]

Is this info that cannot be retrieved using Indeed's API?
https://opensource.indeedeng.io/api-documentation/

There are a few resources on web scraping best-practices/ethics that I link to in the thread here:

website scraping cookies

It looks like they have an API for retrieving data: https://www.marinetraffic.com/en/ais-api-services Though it's not specific to R, this is a nice guide to how to approach web scraping in an ethical way, and you should always try to use an API when it's offered: The polite package helps you do that with R and rvest:

There are also a bunch of great web-scraping posts in hrbrmstr's blog here:

web scraping Archives - rud.is

Including one on scraping when there is a free API available:

More "Scraping Ethics Gone Awry" and "Why Do This When There's a Free API?" -...