I am trying to web scrape job descriptions from the webpage -https://www.indeed.com/cmp/HP/jobs
I am also new to web scrapping and im learning from scratch so I apologize if I am doing something stupid and do not realise it.
I have the following code
test_url <- read_html("https://www.indeed.com/cmp/HP/jobs") job_description <- test_url %>% html_nodes(".cmp-JobDetailDescription") %>% html_text()
I used the CSS selector to find the node but for some reason I cannot get the description.
What do I have to change in my code to extract all the job descriptions in that url?
Someone please help Ive been breaking my head over this for the last few hours
Thank you in advance