webscrapping:: load more

General
1

Hi,

library(rvest)
library(tidyverse)
library(RSelenium)


rD <- rsDriver(verbose = TRUE, 
               port=4812L, 
               chromever = '88.0.4324.27',
               check = TRUE)

url_reviews <- "https://www.airbnb.com/rooms/14421178/reviews?_set_bev_on_new_domain=1613678591_NDFhY2I1MzczM2E3&source_impression_id=p3_1613678592_XuCy0ArGltRr3YTK&guests=1&adults=1&check_in=2021-02-23&check_out=2021-02-24"

remDr <- rD$client

remDr$navigate(url_reviews)

remDr$getPageSource()[[1]] %>% 
  read_html() %>% 
  html_nodes(xpath = '//*[contains(concat( " ", @class, " " ), concat( " ", "_1xib9m0", " " ))]') %>% 
  html_text()

I would like to perform webscrapping on reviews, but I have 2 problems (1) with the load more buttons (if the site appears in hungarian to you, then is the "bővebben" at the end of the comments).
(2) How can you immitate an infinite scrolling down or similar solution? (If you scroll down, you see more data)

Sorry my code doesnt run in reprex :frowning:

Thank you for your answer in advance,
Marcell

2

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.