Adapt code web-scraping script: ignore repeated part reviews

Hello! I'm using a script in RStudio to scrape Goodreads reviews.
If the reviews are long, they are only partially visible, you have to click on "...more" to see the rest of the review:

enter image description here

The script deals with this and "clicks" on the "...more" (if it occurs), so the entire review is scraped:

  #Expand all reviews
  expandMore <- remDr$findElements("link text", "...more")
  sapply(expandMore, function(x) x$clickElement())

  #Extracting the reviews from the page
  reviews <- remDr$findElements("css selector", "#bookReviews .stacked")
  reviews.html <- lapply(reviews, function(x){x$getElementAttribute("outerHTML")[[1]]})
  reviews.list <- lapply(reviews.html, function(x){read_html(x) %>% html_text()} )
  reviews.text <- unlist(reviews.list)

The problem is that part of the review is now scraped twice: first the preview and then the entire review. So what I get is a CSV file containing this kind of reviews (short example I wrote myself):

I really liked the book because of many reasons which will I really liked the book because of many reasons which will be explained in this long review. 
As you can see the first part of my review is repeated.

I would like to change this code, so that the first part of the scraped review is no longer repeated. I want the script to only scrape the full review after "...more" and to ignore or dismiss the preview, but I also still want it to simply scrape the short reviews (without "...more"). So basically I want it to only look at the review after "...more" IF "...more" is present.

I've tried to do it myself by exchanging the line reviews.list <- lapply(reviews.html, function(x){read_html(x) %>% html_text()} ) for reviews.list <- lapply(reviews.html, function(x){ read_html(x) %>% html_nodes("span:last-child") %>% html_text() } ). However, this causes the following error:

Error in UseMethod("xml_find_all") : 
no applicable method for 'xml_find_all' applied to an object of class "character"

I guess there is a clash between html_nodes and span:last-child?

Would someone with more experience in writing R-code and/or web-scraping please help me adapt this part of the script's code? Thank you in advance!

this read_html function, what package is it from ?
it appears to be giving you a character response instead of some kind of xml response that you might further parse with selectors etc.

Hello @nirgrahamuk, thank you for replying to my post! It's from the rvest package (https://cran.r-project.org/web/packages/rvest/rvest.pdf). I'm unsure why it causes a problem in the second line of code, because it doesn't give me an error when I use reviews.list <- lapply(reviews.html, function(x){read_html(x) %>% html_text()} ). What do you think?

this doesn't produce meaningful output, but neither does it error:

reviews.list <- lapply("<html><title>Hi<title></html>", function(x){ read_html(x) %>% html_nodes("span:last-child") %>% html_text() } )

so either there is an issue with your read_html perhaps a namespace conflict.
try

getAnywhere(read_html)

or there is a perculiarity with your data reviews.html which I'd need to get hold of a sample of in order to understand.

@nirgrahamuk Here is the full script, if this helps? You should only need to set your working directory.

library(rJava)        # Required to use RSelenium
library(data.table)   # Required for rbindlist
library(dplyr)        # Required to use the pipes %>% and some table manipulation commands
library(magrittr)     # Required to use the pipes %>%
library(rvest)        # Required for read_html
library(RSelenium)    # Required for webscraping with javascript
library(lubridate)    # Required to scrape the correct dates
library(stringr)      # Required to cut off any leading or trailing whitespace from text
library(purrr)


options(stringsAsFactors = F) #needed to prevent errors when merging data frames

#Paste the GoodReads Url
url <- "https://www.goodreads.com/book/show/38447.The_Handmaid_s_Tale?ac=1&from_search=true&qid=ZGrzc7AfLN&rank=1"

englishOnly = F #If FALSE, all languages are chosen

#Set your browser settings
#Do NOT use Firefox instead! The script will keep on re-scraping the same pages
rD <- rsDriver(port = 5353L, browser = "chrome", chromever = "83.0.4103.39")
remDr <- rD[["client"]]
remDr$setTimeout(type = "implicit", 2000)
remDr$navigate(url)

bookTitle = unlist(remDr$getTitle())
finalData = data.frame()

# Main loop going through the website pages
morePages = T
pageNumber =  1
while(morePages){
  
  #Select reviews in correct language
  selectLanguage = if(englishOnly){
    selectLanguage = remDr$findElement("xpath", "//select[@id='language_code']/option[@value='']")
  } else {
    selectLanguage = remDr$findElement("xpath", "//select[@id='language_code']/option[1]")
  }
  
  selectLanguage$clickElement()
  Sys.sleep(3)
  
  #Expand all reviews
  expandMore <- remDr$findElements("link text", "...more")
  sapply(expandMore, function(x) x$clickElement())
  
  #Extracting the reviews from the page
  reviews <- remDr$findElements("css selector", "#bookReviews .stacked")
  reviews.html <- lapply(reviews, function(x){x$getElementAttribute("outerHTML")[[1]]})
  reviews.list <- lapply(reviews.html, function(x){read_html(x) %>% html_text()} )
  reviews.text <- unlist(reviews.list)
  
  #Some reviews have only rating and no text, so we process them separately
  onlyRating = unlist(map(1:length(reviews.text), function(i) str_detect(reviews.text[i], "^\\\n\\\n")))
  
  #Full reviews
  if(sum(!onlyRating) > 0){
    
    filterData = reviews.text[!onlyRating]
    fullReviews = purrr::map_df(seq(1, length(filterData), by=2), function(i){
      review = unlist(strsplit(filterData[i], "\n"))
      
      data.frame(
        date = mdy(review[2]), #date
        username = str_trim(review[5]), #user
        rating = str_trim(review[9]), #overall
        comment = str_trim(review[12]) #comment
      )
    })
    
    #Add review text to full reviews
    fullReviews$review = unlist(purrr::map(seq(2, length(filterData), by=2), function(i){
      str_trim(str_remove(filterData[i], "\\s*\\n\\s*\\(less\\)"))
    }))
    
  } else {
    fullReviews = data.frame()
  }
  
  #partial reviews (only rating)
  if(sum(onlyRating) > 0){
    
    filterData = reviews.text[onlyRating]
    partialReviews = purrr::map_df(1:length(filterData), function(i){
      review = unlist(strsplit(filterData[i], "\n"))
      
      data.frame(
        date = mdy(review[9]), #date
        username = str_trim(review[4]), #user
        rating = str_trim(review[8]), #overall
        comment = "",
        review = ""
      )
    })
    
  } else {
    partialReviews = data.frame()
  }
  
  #Get the review ID's from all the links
  reviewId = reviews.html %>% str_extract("/review/show/\\d+")
  partialId = reviewId[(length(reviewId) - nrow(partialReviews) + 1):length(reviewId)]
  reviewId = reviewId[1:(length(reviewId) - nrow(partialReviews))]
  reviewId = reviewId[seq(1, length(reviewId), 2)] %>% str_extract("\\d+")
  if(nrow(partialReviews) > 0){
    reviewId = c(reviewId, partialId)
  }
  
  finalData = rbind(finalData, cbind(reviewId, rbind(fullReviews, partialReviews)))
  
  #Go to next page if possible
  nextPage = remDr$findElements("xpath", "//a[@class='next_page']")
  if(length(nextPage) > 0){
    message(paste("PAGE", pageNumber, "Processed - Going to next"))
    nextPage[[1]]$clickElement()
    pageNumber = pageNumber + 1
    Sys.sleep(2)
  } else {
    message(paste("PAGE", pageNumber, "Processed - Last page"))
    morePages = FALSE
  }
  
}   
#end of the main loop

#Replace missing ratings by 'not rated'
finalData$rating = ifelse(finalData$rating == "", "not rated", finalData$rating)

#Stop server
remDr$close()
rD$server$stop()

#set directory to where you wish the file to go
#copy your working directory and exchange all backward slashes for forward slashes
getwd()
setwd("")

#Write results
write.csv(finalData, paste0(bookTitle, ".csv"), row.names = F)
message("FINISHED!")

sorry, but what you've shared is not a minimal reproducible example, ... there is a lot of code there.

I personally don't want rJava on my machine, so I wont be using Rselenium.

However, if you can run your code to the point where there is data saved into reviews.html for one of your pages. you can share that data with the dput() function. and we can discuss the code that would immediately be processing that shared object that we would both have.

I'm sorry, I have no idea how to create a minimal reproducible example - I'm rather new at this and don't have a lot of experience with scripts. I think you might be able to run it without rJava, I was able to until my computer started acting strange after an update, that's when I had to install it, before I was fine without it.
Where should i put dput(reviews.html)?
Thank you for helping!

you would do this in your console. but it assumes there is a review.html object that you made already.
to read about reprex see

@nirgrahamuk Hello again, I ran it in my console and got the following output:

list("<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/2162483563?book_show_action=false\">Oct 24, 2017</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"TeaAndBooks\" class=\"user\" itemprop=\"url\" name=\"TeaAndBooks\" href=\"/user/show/72062985-teaandbooks\">TeaAndBooks</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"it was amazing\"><span size=\"15x15\" class=\"staticStar p10\">it was amazing</span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span></span>\n\n\n\n        \n\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer2162483563\" class=\"readable\">\n            \n<span id=\"freeTextContainer8146964391255470601\" style=\"display: none;\">\" A rat in a maze is free to go anywhere, as long as it stays inside the maze.\" <br><br>Incredible, simply incredible. <br>I was speechless at the plot and the main concept of the story. I haven't read a large amount of dystopians but this book, in my opinion, would still be my favourite. It left me speechless. <br>I loved the idea of the story and it was so thought-provoking! <br>It's like a reminder of what society could become like and that leaves the reader thinking.<br>The ending was a huge cliffhanger and I want </span>\n  <span id=\"freeText8146964391255470601\" style=\"\">\" A rat in a maze is free to go anywhere, as long as it stays inside the maze.\" <br><br>Incredible, simply incredible. <br>I was speechless at the plot and the main concept of the story. I haven't read a large amount of dystopians but this book, in my opinion, would still be my favourite. It left me speechless. <br>I loved the idea of the story and it was so thought-provoking! <br>It's like a reminder of what society could become like and that leaves the reader thinking.<br>The ending was a huge cliffhanger and I want more. <br>Overall, an amazing book! </span>\n  <a data-text-id=\"8146964391255470601\" href=\"#\" onclick=\"swapContent($(this));; return false;\">(less)</a>\n\n          </span>\n        </div>", 
    "<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/989357675?book_show_action=false\">Jul 07, 2014</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Andrei Florian\" class=\"user\" itemprop=\"url\" name=\"Andrei Florian\" href=\"/user/show/25947039-andrei-florian\">Andrei Florian</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"really liked it\"><span size=\"15x15\" class=\"staticStar p10\">really liked it</span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p0\"></span></span>\n\n\n\n        \n\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer989357675\" class=\"readable\">\n            \n<span id=\"freeTextContainer5046653536476187020\">Recomand Povestirea cameristei de Margaret Atwood ce face parte din genul de \"fictiune speculativa\" - in care se prezinta eludarea personalitatii feminine, pana la reducerea la simple obiecte de reproducere a rasei umane (scriitoarea aminteste chiar de Decretul antiavorturi din Romania anului 1966 pentru cresterea numarului populatiei), intr-o societate anarhica, dominata de utopia patriarhatului de tip feudal, in numele credintei unei anumite secte religioase.</span>\n\n          </span>\n        </div>", 
    "<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/1808353313?book_show_action=false\">Nov 14, 2016</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Mary\" class=\"user\" itemprop=\"url\" name=\"Mary\" href=\"/user/show/5304930-mary\">Mary</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"it was amazing\"><span size=\"15x15\" class=\"staticStar p10\">it was amazing</span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span></span>\n\n\n\n        \n\n          <div class=\"uitext greyText bookshelves\">\n            Shelves:\n              <a class=\"actionLinkLite\" href=\"/review/list/5304930-mary?shelf=fiction\">fiction</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/5304930-mary?shelf=2016\">2016</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/5304930-mary?shelf=surviving-fuhrer-trump\">surviving-fuhrer-trump</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/5304930-mary?shelf=dystopia\">dystopia</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/5304930-mary?shelf=2019\">2019</a>\n          </div>\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer1808353313\" class=\"readable\">\n            \n<span id=\"freeTextContainer4185930522659220636\">(He) was of the opinion from the outset that the best and most cost-effective way to control women for reproductive and other purposes was through women themselves. For this there were many historical precedents; in fact, no empire imposed by force or otherwise has ever been without this feature: control of the indigenous by members of their own group.<br><br>(p. 308)</span>\n\n          </span>\n        </div>", 
    "<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/1401460137?book_show_action=false\">Feb 11, 2017</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Solistas\" class=\"user\" itemprop=\"url\" name=\"Solistas\" href=\"/user/show/42053546-solistas\">Solistas</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"really liked it\"><span size=\"15x15\" class=\"staticStar p10\">really liked it</span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p0\"></span></span>\n\n          <span class=\"greyText\">&nbsp;·&nbsp;</span>\n          <a class=\"lightGreyText\" title=\"Paperback 9780771008795\" href=\"/book/show/12590320-the-handmaid-s-tale\">review of another edition</a>\n\n\n        \n\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer1401460137\" class=\"readable\">\n            \n<span id=\"freeTextContainer9987639919750791864\" style=\"display: none;\"><U+039C><U+03B9>a p<U+03BF><U+03BB><U+03CD> d<U+03C5><U+03BD>at<U+03AE> <U+03B9>st<U+03BF><U+03C1><U+03AF>a ap<U+03CC> µ<U+03B9>a p<U+03BF><U+03BB><U+03CD> d<U+03C5><U+03BD>at<U+03AE> p<U+03AD><U+03BD>a. <U+0397> <U+03B9>st<U+03BF><U+03C1><U+03AF>a t<U+03B7><U+03C2> p<U+03BF><U+03C1>f<U+03C5><U+03C1><U+03AE><U+03C2> d<U+03BF><U+03CD><U+03BB><U+03B7><U+03C2> <U+03CC>p<U+03C9><U+03C2> t<U+03BF> \"µet<U+03AD>f<U+03C1>ase\" <U+03BF> <U+039C><U+03AC>tes<U+03B9><U+03C2> e<U+03AF><U+03BD>a<U+03B9> µ<U+03B9>a <U+03AC><U+03BA><U+03C1><U+03C9><U+03C2> <U+03C1>ea<U+03BB><U+03B9>st<U+03B9><U+03BA><U+03AE> d<U+03C5>st<U+03BF>p<U+03AF>a p<U+03BF><U+03C5> af<U+03B7><U+03B3>e<U+03AF>ta<U+03B9> µ<U+03B9>a <U+03B3><U+03C5><U+03BD>a<U+03AF><U+03BA>a p<U+03BF><U+03C5> <U+03AD><U+03C7>e<U+03B9> <U+03C7><U+03AC>se<U+03B9> ta p<U+03AC><U+03BD>ta. <U+038C><U+03C7><U+03B9> µ<U+03CC><U+03BD><U+03BF> t<U+03B7> <U+03B6><U+03C9><U+03AE> t<U+03B7><U+03C2> <U+03CC>p<U+03C9><U+03C2> t<U+03B7><U+03BD> <U+03B3><U+03BD><U+03CE><U+03C1><U+03B9><U+03B6>e a<U+03BB><U+03BB><U+03AC> <U+03BA> t<U+03BF> s<U+03CE>µa t<U+03B7><U+03C2> p<U+03BF><U+03C5> t<U+03CE><U+03C1>a <U+03B3><U+03AF><U+03BD>eta<U+03B9> ap<U+03BB><U+03AC> <U+03AD><U+03BD>a s<U+03BA>e<U+03CD><U+03BF><U+03C2> p<U+03BF><U+03C5> <U+03CC>s<U+03BF> µp<U+03BF><U+03C1>e<U+03AF> <U+03BD>a te<U+03BA><U+03BD><U+03BF>p<U+03BF><U+03B9><U+03AE>se<U+03B9> t<U+03B7><U+03BD> <U+03BA><U+03C1>at<U+03AC>e<U+03B9> st<U+03B7> <U+03B6><U+03C9><U+03AE>. <U+0397> Atwood e<U+03AF><U+03BD>a<U+03B9> e<U+03BE>a<U+03B9><U+03C1>et<U+03B9><U+03BA><U+03AE> s<U+03C5><U+03B3><U+03B3><U+03C1>af<U+03AD>a<U+03C2>, <U+03BA> <U+03CC>sa <U+03BA> a<U+03BD> µp<U+03BF><U+03C1>e<U+03AF> <U+03BA><U+03AC>p<U+03BF><U+03B9><U+03BF><U+03C2> <U+03BD>a p<U+03C1><U+03BF>s<U+03AC><U+03C8>e<U+03B9> st<U+03B7><U+03BD> <U+03AF>d<U+03B9>a t<U+03B7><U+03BD> <U+03B9>st<U+03BF><U+03C1><U+03AF>a pa<U+03C1>aµe<U+03C1><U+03AF><U+03B6><U+03BF><U+03BD>ta<U+03B9>, <U+03B3><U+03B9>at<U+03AF> <U+03B7> a<U+03BD><U+03AC><U+03B3><U+03BD><U+03C9>s<U+03B7> t<U+03BF><U+03C5> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF><U+03C5> e<U+03AF><U+03BD>a<U+03B9> <U+03BA>a<U+03B8><U+03B7><U+03BB><U+03C9>t<U+03B9><U+03BA><U+03AE> <U+03BA> </span>\n  <span id=\"freeText9987639919750791864\" style=\"\"><U+039C><U+03B9>a p<U+03BF><U+03BB><U+03CD> d<U+03C5><U+03BD>at<U+03AE> <U+03B9>st<U+03BF><U+03C1><U+03AF>a ap<U+03CC> µ<U+03B9>a p<U+03BF><U+03BB><U+03CD> d<U+03C5><U+03BD>at<U+03AE> p<U+03AD><U+03BD>a. <U+0397> <U+03B9>st<U+03BF><U+03C1><U+03AF>a t<U+03B7><U+03C2> p<U+03BF><U+03C1>f<U+03C5><U+03C1><U+03AE><U+03C2> d<U+03BF><U+03CD><U+03BB><U+03B7><U+03C2> <U+03CC>p<U+03C9><U+03C2> t<U+03BF> \"µet<U+03AD>f<U+03C1>ase\" <U+03BF> <U+039C><U+03AC>tes<U+03B9><U+03C2> e<U+03AF><U+03BD>a<U+03B9> µ<U+03B9>a <U+03AC><U+03BA><U+03C1><U+03C9><U+03C2> <U+03C1>ea<U+03BB><U+03B9>st<U+03B9><U+03BA><U+03AE> d<U+03C5>st<U+03BF>p<U+03AF>a p<U+03BF><U+03C5> af<U+03B7><U+03B3>e<U+03AF>ta<U+03B9> µ<U+03B9>a <U+03B3><U+03C5><U+03BD>a<U+03AF><U+03BA>a p<U+03BF><U+03C5> <U+03AD><U+03C7>e<U+03B9> <U+03C7><U+03AC>se<U+03B9> ta p<U+03AC><U+03BD>ta. <U+038C><U+03C7><U+03B9> µ<U+03CC><U+03BD><U+03BF> t<U+03B7> <U+03B6><U+03C9><U+03AE> t<U+03B7><U+03C2> <U+03CC>p<U+03C9><U+03C2> t<U+03B7><U+03BD> <U+03B3><U+03BD><U+03CE><U+03C1><U+03B9><U+03B6>e a<U+03BB><U+03BB><U+03AC> <U+03BA> t<U+03BF> s<U+03CE>µa t<U+03B7><U+03C2> p<U+03BF><U+03C5> t<U+03CE><U+03C1>a <U+03B3><U+03AF><U+03BD>eta<U+03B9> ap<U+03BB><U+03AC> <U+03AD><U+03BD>a s<U+03BA>e<U+03CD><U+03BF><U+03C2> p<U+03BF><U+03C5> <U+03CC>s<U+03BF> µp<U+03BF><U+03C1>e<U+03AF> <U+03BD>a te<U+03BA><U+03BD><U+03BF>p<U+03BF><U+03B9><U+03AE>se<U+03B9> t<U+03B7><U+03BD> <U+03BA><U+03C1>at<U+03AC>e<U+03B9> st<U+03B7> <U+03B6><U+03C9><U+03AE>. <U+0397> Atwood e<U+03AF><U+03BD>a<U+03B9> e<U+03BE>a<U+03B9><U+03C1>et<U+03B9><U+03BA><U+03AE> s<U+03C5><U+03B3><U+03B3><U+03C1>af<U+03AD>a<U+03C2>, <U+03BA> <U+03CC>sa <U+03BA> a<U+03BD> µp<U+03BF><U+03C1>e<U+03AF> <U+03BA><U+03AC>p<U+03BF><U+03B9><U+03BF><U+03C2> <U+03BD>a p<U+03C1><U+03BF>s<U+03AC><U+03C8>e<U+03B9> st<U+03B7><U+03BD> <U+03AF>d<U+03B9>a t<U+03B7><U+03BD> <U+03B9>st<U+03BF><U+03C1><U+03AF>a pa<U+03C1>aµe<U+03C1><U+03AF><U+03B6><U+03BF><U+03BD>ta<U+03B9>, <U+03B3><U+03B9>at<U+03AF> <U+03B7> a<U+03BD><U+03AC><U+03B3><U+03BD><U+03C9>s<U+03B7> t<U+03BF><U+03C5> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF><U+03C5> e<U+03AF><U+03BD>a<U+03B9> <U+03BA>a<U+03B8><U+03B7><U+03BB><U+03C9>t<U+03B9><U+03BA><U+03AE> <U+03BA> de<U+03BD> t<U+03BF> <U+03BA>ataf<U+03AD><U+03C1><U+03BD><U+03BF><U+03C5><U+03BD> a<U+03C5>t<U+03CC> <U+03BA> p<U+03BF><U+03BB><U+03BB><U+03AC> ß<U+03B9>ß<U+03BB><U+03AF>a. <br><br><U+0395>pe<U+03B9>d<U+03AE> fa<U+03AF><U+03BD>eta<U+03B9> <U+03CC>t<U+03B9> <U+03CC><U+03BB><U+03BF><U+03B9> t<U+03BF> <U+03AD><U+03C7><U+03BF><U+03C5><U+03BD> d<U+03B9>aß<U+03AC>se<U+03B9> <U+03AE> s<U+03BA><U+03BF>pe<U+03CD><U+03BF><U+03C5><U+03BD> <U+03BD>a t<U+03BF> <U+03BA><U+03AC><U+03BD><U+03BF><U+03C5><U+03BD> (e<U+03B9>d<U+03B9><U+03BA><U+03AC> t<U+03CE><U+03C1>a p<U+03BF><U+03C5> t<U+03BF> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF> µetaf<U+03AD><U+03C1>eta<U+03B9> st<U+03B7><U+03BD> µ<U+03B9><U+03BA><U+03C1><U+03AE> <U+03BF><U+03B8><U+03CC><U+03BD><U+03B7> <U+03BA> st<U+03B7><U+03BD> <U+03BF>µ<U+03CE><U+03BD><U+03C5>µ<U+03B7> se<U+03B9><U+03C1><U+03AC> p<U+03BF><U+03C5> <U+03BE>e<U+03BA><U+03B9><U+03BD><U+03AC>e<U+03B9> t<U+03AD><U+03BB><U+03B7> <U+0391>p<U+03C1><U+03AF><U+03BB><U+03B7>) de<U+03BD> <U+03C5>p<U+03AC><U+03C1><U+03C7>e<U+03B9> <U+03BB><U+03CC><U+03B3><U+03BF><U+03C2> <U+03BD>a pe<U+03C1><U+03B9><U+03B3><U+03C1>afe<U+03AF> <U+03B7> <U+03B9>st<U+03BF><U+03C1><U+03AF>a p<U+03AD><U+03C1>a ap'ta ßas<U+03B9><U+03BA><U+03AC>. <U+0397> <U+03CC><U+03C7><U+03B9> <U+03BA> t<U+03CC>s<U+03BF> µe<U+03BB><U+03BB><U+03BF><U+03BD>t<U+03B9><U+03BA><U+03AE> <U+0391>µe<U+03C1><U+03B9><U+03BA><U+03AE> µet<U+03BF><U+03BD><U+03BF>µ<U+03AC><U+03B6>eta<U+03B9> se d<U+03B7>µ<U+03BF><U+03BA><U+03C1>at<U+03AF>a t<U+03BF><U+03C5> Gilead <U+03CC>p<U+03BF><U+03C5> <U+03BF><U+03B9> <U+03B3><U+03C5><U+03BD>a<U+03AF><U+03BA>e<U+03C2> <U+03C7><U+03AC><U+03BD><U+03BF><U+03C5><U+03BD> <U+03BA><U+03AC><U+03B8>e e<U+03BB>e<U+03C5><U+03B8>e<U+03C1><U+03AF>a <U+03BA> d<U+03B9>a<U+03C7><U+03C9><U+03C1><U+03AF><U+03B6><U+03BF><U+03BD>ta<U+03B9> µe ß<U+03AC>s<U+03B7> t<U+03B7><U+03BD> a<U+03BD>apa<U+03C1>a<U+03B3><U+03C9><U+03B3><U+03B9><U+03BA><U+03AE> t<U+03BF><U+03C5><U+03C2> <U+03B9><U+03BA>a<U+03BD><U+03CC>t<U+03B7>ta. <U+0397> <U+03B8>e<U+03BF><U+03BA><U+03C1>at<U+03B9><U+03BA><U+03AE> d<U+03B7>µ<U+03BF><U+03BA><U+03C1>at<U+03AF>a t<U+03BF><U+03C5><U+03C2> pa<U+03AF><U+03C1><U+03BD>e<U+03B9> a<U+03C5>t<U+03BF><U+03BB>e<U+03BE>e<U+03AF> t<U+03BF> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF> t<U+03B7><U+03C2> G<U+03AD><U+03BD>es<U+03B9><U+03C2> <U+03BA> t<U+03BF> efa<U+03C1>µ<U+03CC><U+03B6>e<U+03B9> µe µe<U+03B3><U+03AC><U+03BB><U+03B7> a<U+03C5>st<U+03B7><U+03C1><U+03CC>t<U+03B7>ta. <U+03A4><U+03BF> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF> e<U+03AF><U+03BD>a<U+03B9> p<U+03C1><U+03CE>ta <U+03BA> <U+03BA><U+03CD><U+03C1><U+03B9>a feµ<U+03B9><U+03BD><U+03B9>st<U+03B9><U+03BA><U+03CC> a<U+03BB><U+03BB><U+03AC> a<U+03BA><U+03BF><U+03C5>µp<U+03AC>e<U+03B9> d<U+03B9>e<U+03BE><U+03BF>d<U+03B9><U+03BA><U+03AC> <U+03BA><U+03B9> <U+03AC><U+03BB><U+03BB>a <U+03B8><U+03AD>µata, <U+03B3><U+03B9>a t<U+03B7> se<U+03BE><U+03BF><U+03C5>a<U+03BB><U+03B9><U+03BA><U+03CC>t<U+03B7>ta, t<U+03B7> µ<U+03BF><U+03BD>a<U+03BE><U+03B9><U+03AC>, t<U+03B7><U+03BD> <U+03BF><U+03B9><U+03BA><U+03BF><U+03B3><U+03AD><U+03BD>e<U+03B9>a, t<U+03B7><U+03BD> e<U+03BE><U+03BF><U+03C5>s<U+03AF>a, t<U+03B7><U+03BD> e<U+03C5><U+03B8><U+03CD><U+03BD><U+03B7> <U+03B3><U+03B9>a a<U+03BD>t<U+03AF>stas<U+03B7> <U+03BA><U+03BF><U+03BA>. <U+0397> Atwood <U+03AD><U+03C7>e<U+03B9> t<U+03BF><U+03BD> ap<U+03CC><U+03BB><U+03C5>t<U+03BF> <U+03AD><U+03BB>e<U+03B3><U+03C7><U+03BF> t<U+03BF><U+03C5> a<U+03BD>a<U+03B3><U+03BD><U+03CE>st<U+03B7> p<U+03BF><U+03C5> pa<U+03C1>a<U+03BA><U+03BF><U+03BB><U+03BF><U+03C5><U+03B8>e<U+03AF> µe <U+03AD><U+03BD>tas<U+03B7> t<U+03B9><U+03C2> s<U+03BA><U+03AD><U+03C8>e<U+03B9><U+03C2> t<U+03B7><U+03C2> af<U+03B7><U+03B3><U+03AE>t<U+03C1><U+03B9>a<U+03C2> a<U+03BA><U+03CC>µa <U+03BA><U+03B9> <U+03CC>ta<U+03BD> <U+03B7> e<U+03BE><U+03AD><U+03BB><U+03B9><U+03BE><U+03B7> t<U+03B7><U+03C2> p<U+03BB><U+03BF><U+03BA><U+03AE><U+03C2> <U+03C1><U+03AF><U+03C7><U+03BD>e<U+03B9> t<U+03BF><U+03C5><U+03C2> <U+03C1><U+03C5><U+03B8>µ<U+03BF><U+03CD><U+03C2> t<U+03B7><U+03C2> <U+03BA><U+03AC>p<U+03BF><U+03C5> sta µ<U+03B9>s<U+03AC> t<U+03BF><U+03C5> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF><U+03C5>. <U+0397> <U+03BA><U+03BF><U+03C1><U+03CD>f<U+03C9>s<U+03B7> t<U+03BF><U+03C5> f<U+03B9><U+03BD><U+03AC><U+03BB>e de<U+03BD> µe <U+03B9><U+03BA>a<U+03BD><U+03BF>p<U+03BF><U+03AF><U+03B7>se <U+03BA>a<U+03B8><U+03CC><U+03BB><U+03BF><U+03C5>, µ<U+03BF><U+03C5> f<U+03AC><U+03BD><U+03B7><U+03BA>e <U+03CC>t<U+03B9> <U+03B7> s<U+03C5><U+03B3><U+03B3><U+03C1>af<U+03AD>a<U+03C2> µ<U+03BF><U+03C5> f<U+03AD><U+03C1><U+03B8><U+03B7><U+03BA>e <U+03BB><U+03AF><U+03B3><U+03BF> sa<U+03BD> <U+03BD>a e<U+03AF>µa<U+03B9> <U+03C7>a<U+03B6><U+03CC><U+03C2> <U+03B3><U+03B9>'a<U+03C5>t<U+03CC> <U+03BA> <U+03AD>p<U+03C1>epe st<U+03BF> t<U+03AD><U+03BB><U+03BF><U+03C2> <U+03BD>a t<U+03BF><U+03BD><U+03AF>se<U+03B9> a<U+03BA><U+03CC>µa p<U+03B9><U+03BF> <U+03AD><U+03BD>t<U+03BF><U+03BD>a <U+03CC>sa <U+03AE><U+03B8>e<U+03BB>e <U+03BD>a pe<U+03B9>. <U+0391><U+03BA><U+03BF>µ<U+03AC> <U+03BA><U+03B9> <U+03AD>ts<U+03B9> <U+03CC>µ<U+03C9><U+03C2>, <U+03B7> <U+03B3>e<U+03CD>s<U+03B7> p<U+03BF><U+03C5> s<U+03BF><U+03C5> af<U+03AE><U+03BD>e<U+03B9> st<U+03BF> st<U+03CC>µa t<U+03BF> <U+03BA>e<U+03AF>µe<U+03BD><U+03BF> de<U+03BD> <U+03C7>a<U+03BB><U+03AC>e<U+03B9>.<br><br><U+03A4><U+03BF> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF> t<U+03BF> d<U+03B9><U+03AC>ßasa pa<U+03C1><U+03AC><U+03BB><U+03BB><U+03B7><U+03BB>a µe t<U+03B7><U+03BD> µet<U+03AC>f<U+03C1>as<U+03B7> t<U+03BF><U+03C5> <U+039C><U+03AC>tes<U+03B7>, <U+03B7> <U+03BF>p<U+03BF><U+03AF>a ß<U+03AD>ßa<U+03B9>a de<U+03BD> e<U+03AF><U+03BD>a<U+03B9> µet<U+03AC>f<U+03C1>as<U+03B7> a<U+03BB><U+03BB><U+03AC> d<U+03B9>as<U+03BA>e<U+03C5><U+03AE> af<U+03BF><U+03CD> ep<U+03AF> t<U+03B7><U+03C2> <U+03BF><U+03C5>s<U+03AF>a<U+03C2> t<U+03BF> a<U+03C5><U+03B8>e<U+03BD>t<U+03B9><U+03BA><U+03CC> <U+03BA>e<U+03AF>µe<U+03BD><U+03BF> e<U+03AF><U+03BD>a<U+03B9> <U+03BA><U+03AC>t<U+03B9> te<U+03BB>e<U+03AF><U+03C9><U+03C2> d<U+03B9>af<U+03BF><U+03C1>et<U+03B9><U+03BA><U+03CC> ap<U+03CC> a<U+03C5>t<U+03CC> p<U+03BF><U+03C5> <U+03B8>a d<U+03B9>aß<U+03AC>se<U+03B9> <U+03CC>p<U+03BF><U+03B9><U+03BF><U+03C2> ep<U+03B9><U+03BB><U+03AD><U+03BE>e<U+03B9> t<U+03B7><U+03BD> e<U+03BB><U+03BB><U+03B7><U+03BD><U+03B9><U+03BA><U+03AE> <U+03AD><U+03BA>d<U+03BF>s<U+03B7>. <U+0394>e <U+03BD><U+03BF>µ<U+03AF><U+03B6><U+03C9> <U+03BD>a <U+03AD><U+03C7><U+03C9> <U+03BE>a<U+03BD>as<U+03C5><U+03BD>a<U+03BD>t<U+03AE>se<U+03B9> p<U+03AC><U+03BB><U+03B9> <U+03BA><U+03AC>t<U+03B9> pa<U+03C1><U+03CC>µ<U+03BF><U+03B9><U+03BF>, <U+03BF> µetaf<U+03C1>ast<U+03AE><U+03C2> <U+03BA><U+03B9><U+03BD>e<U+03AF>ta<U+03B9> µe ap<U+03AF>ste<U+03C5>t<U+03B7> e<U+03BB>e<U+03C5><U+03B8>e<U+03C1><U+03AF>a p<U+03AC><U+03BD><U+03C9> st<U+03BF> <U+03BA>e<U+03AF>µe<U+03BD><U+03BF> p<U+03BF><U+03C5> <U+03B3><U+03B9>a <U+03BA><U+03AC>p<U+03BF><U+03B9><U+03BF> <U+03BB><U+03CC><U+03B3><U+03BF> <U+03B8>e<U+03CE><U+03C1><U+03B7>se s<U+03C9>st<U+03CC> <U+03BD>a <U+03BA><U+03AC><U+03BD>e<U+03B9> te<U+03BB>e<U+03AF><U+03C9><U+03C2> d<U+03B9><U+03BA><U+03CC> t<U+03BF><U+03C5>. <U+038C>s<U+03BF><U+03B9> <U+03AD><U+03C7><U+03BF><U+03C5><U+03BD> d<U+03B9>aß<U+03AC>se<U+03B9> <U+039C><U+03AC>tes<U+03B7> <U+03B8>a t<U+03BF> <U+03BA>ata<U+03BB><U+03AC>ß<U+03BF><U+03C5><U+03BD>. <U+03A0><U+03C1><U+03BF>s<U+03B8><U+03AD>te<U+03B9> <U+03BF><U+03BB><U+03CC><U+03BA><U+03BB><U+03B7><U+03C1>e<U+03C2> p<U+03C1><U+03BF>t<U+03AC>se<U+03B9><U+03C2> <U+03CC>ta<U+03BD> <U+03BD><U+03BF>µ<U+03AF><U+03B6>e<U+03B9> <U+03CC>t<U+03B9> <U+03BA><U+03AC>t<U+03B9> de<U+03BD> e<U+03AF><U+03BD>a<U+03B9> saf<U+03AD><U+03C2>, <U+03BA><U+03CC>ße<U+03B9> <U+03CC>ta<U+03BD> a<U+03BD>t<U+03AF>st<U+03BF><U+03B9><U+03C7>a p<U+03B9>ste<U+03CD>e<U+03B9> <U+03CC>t<U+03B9> <U+03BA><U+03AC>t<U+03B9> de<U+03BD> <U+03C7><U+03C1><U+03B7>s<U+03B9>µe<U+03CD>e<U+03B9> <U+03B9>d<U+03B9>a<U+03AF>te<U+03C1>a, se <U+03B3>e<U+03BD><U+03B9><U+03BA><U+03AD><U+03C2> <U+03B3><U+03C1>aµµ<U+03AD><U+03C2> p<U+03C1><U+03CC><U+03BA>e<U+03B9>ta<U+03B9> <U+03B3><U+03B9>a d<U+03CD><U+03BF> d<U+03B9>af<U+03BF><U+03C1>et<U+03B9><U+03BA><U+03AC> ß<U+03B9>ß<U+03BB><U+03AF>a p<U+03BF><U+03C5> p<U+03C1><U+03BF>spa<U+03B8><U+03BF><U+03CD><U+03BD> <U+03BD>a p<U+03BF><U+03C5><U+03BD> t<U+03B7><U+03BD> <U+03AF>d<U+03B9>a <U+03B9>st<U+03BF><U+03C1><U+03AF>a. <U+03A7><U+03C1><U+03C9>st<U+03AC><U+03C9> p<U+03BF><U+03BB><U+03BB><U+03AC> st<U+03B7> <U+039C><U+03B7>t<U+03AD><U+03C1>a t<U+03BF><U+03C5> S<U+03BA><U+03CD><U+03BB><U+03BF><U+03C5> sa<U+03BD> a<U+03BD>a<U+03B3><U+03BD><U+03CE>st<U+03B7><U+03C2> <U+03CC>ta<U+03BD> t<U+03B7> d<U+03B9><U+03AC>ßasa <U+03C9><U+03C2> <U+03AD>f<U+03B7>ß<U+03BF><U+03C2> a<U+03BB><U+03BB><U+03AC> a<U+03C5>t<U+03AE> ed<U+03CE> e<U+03AF><U+03BD>a<U+03B9> <U+03B7> <U+03C7>e<U+03B9><U+03C1><U+03CC>te<U+03C1><U+03B7> µet<U+03AC>f<U+03C1>as<U+03B7> p<U+03BF><U+03C5> <U+03AD><U+03C7><U+03C9> s<U+03C5><U+03BD>a<U+03BD>t<U+03AE>se<U+03B9> p<U+03BF>t<U+03AD>.<br><br><U+0391><U+03C2> e<U+03AF><U+03BD>a<U+03B9> <U+03CC>µ<U+03C9><U+03C2>. <U+03A0><U+03C1><U+03CC><U+03BA>e<U+03B9>ta<U+03B9> <U+03B3><U+03B9>a e<U+03BE>a<U+03B9><U+03C1>et<U+03B9><U+03BA><U+03CC> ß<U+03B9>ß<U+03BB><U+03AF><U+03BF> p<U+03BF><U+03C5> <U+03AC><U+03C1><U+03B3><U+03B7>sa p<U+03AC><U+03BD><U+03C9> ap<U+03CC> d<U+03CD><U+03BF> de<U+03BA>aet<U+03AF>e<U+03C2> <U+03BD>a d<U+03B9>aß<U+03AC>s<U+03C9>. Ta ep<U+03B9>st<U+03C1><U+03AD><U+03C8><U+03C9> <U+03BA> st<U+03B7><U+03BD> Atwood µ<U+03AD>sa st<U+03B7> <U+03C7><U+03C1><U+03BF><U+03BD><U+03B9><U+03AC> a<U+03BB><U+03BB><U+03AC> <U+03BA> se <U+03AC><U+03BB><U+03BB>a <U+03BA><U+03BB>ass<U+03B9><U+03BA><U+03AC> d<U+03C5>st<U+03BF>p<U+03B9><U+03BA><U+03AC> <U+03BA>e<U+03AF>µe<U+03BD>a a<U+03BD>aß<U+03AC><U+03BB><U+03BB><U+03C9> s<U+03C5><U+03BD><U+03AD><U+03C7>e<U+03B9>a.</span>\n  <a data-text-id=\"9987639919750791864\" href=\"#\" onclick=\"swapContent($(this));; return false;\">(less)</a>\n\n          </span>\n        </div>")

I did need to shorten it, because it exeeded the character limit. Does this help?

ok thanks for that.
The first thing I noticed is that the 8th review is twice as big as the first 7 views combined.
In fact the line

reviews.list <- lapply(reviews.html, function(x){read_html(x) %>% html_text()} )

failed to run and gave an error

 Error in read_xml.raw(charToRaw(enc2utf8(x)), "UTF-8", ..., as_html = as_html,  : 
  Excessive depth in document: 256 use XML_PARSE_HUGE option [1] 

did you notice this behaviour ?

it can be fixed by

reviews.list <- lapply(reviews.html, function(x){read_html(x,options = "HUGE") %>% html_text()} )

@nirgrahamuk
Thank you for pointing it out! I must admit I never noticed that behaviour. I usually just run the script in one go in RStudio and then I simply get this output in the console:

PAGE 1 Processed - Going to next
PAGE 2 Processed - Going to next
PAGE 3 Processed - Going to next
PAGE 4 Processed - Going to next
PAGE 5 Processed - Going to next
PAGE 6 Processed - Going to next
PAGE 7 Processed - Going to next
PAGE 8 Processed - Going to next
PAGE 9 Processed - Going to next
PAGE 10 Processed - Last page

and then the csv-file is in the folder.

I've run the script again with the changed line and have compared the two resulting csv-files, but there doesn't appear to be a difference.

alright ! moving on...
so the issue was flagged was that adding a span:last-child step caused an error.
I tried this

lapply(reviews.html, function(x){read_html(x,options = "HUGE")  %>% html_nodes("span:last-child")  %>% html_text() } )

I didnt get any error there with the reviews.html that was shared to me. with that same reviews.html do you get the error you had originally ?

@nirgrahamuk Thank you for helping me!
I replaced the line of code with the one you suggested and I got the following error message:

Error in `$<-.data.frame`(`*tmp*`, "review", value = c("One of these books that make you think: \"wow, you can write like that???\". Very inspiring. It's a collection of snippets from a weird, neo-futurist city where cyberfish coexist with electronymphs and robot keepers of the wind, and where the human is judged by trees. A hypnotising piece of good poetry.",  : 
  replacement has 4 rows, data has 5

The csv-file that was created was empty, so I don't think it succeeded in scraping reviews anymore.

I feel like youve jumped ahead, you didnt answer the question I asked....

@nirgrahamuk Oh, I'm sorry! I ran dput(reviews.html) again and received the following output:

list("<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/160930094?book_show_action=false\">Apr 13, 2011</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Kate\" class=\"user\" itemprop=\"url\" name=\"Kate\" href=\"/user/show/4482859-kate\">Kate</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"did not like it\"><span size=\"15x15\" class=\"staticStar p10\">did not like it</span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span></span>\n\n          <span class=\"greyText\">&nbsp;·&nbsp;</span>\n          <a class=\"lightGreyText\" title=\"Kindle Edition \" href=\"/book/show/7439970-the-handmaid-s-tale\">review of another edition</a>\n\n\n        \n\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer160930094\" class=\"readable\">\n            \n<span id=\"freeTextContainer1606444764647290275\" style=\"display: none;\">\n  <em>It's been almost five years since I wrote my review. I've rewritten large parts of it for clarity. The main idea remains the same.</em>\n<br><br>Extremist Judeo-Christian beliefs have won America's culture war. Now women have no rights. They are slaves to men and the biblical, patriarchal society in which they live. <em>The Handmaid's Tale</em> is the first-person account of one of these enslaved women.<br><br><strong>Massachusetts Turns Into Saudi Arabia?</strong><br>More than thirty years have passed since <em>The Handmaid's Tale</em> was first publi</span>\n  <span id=\"freeText1606444764647290275\" style=\"\">\n  <em>It's been almost five years since I wrote my review. I've rewritten large parts of it for clarity. The main idea remains the same.</em>\n<br><br>Extremist Judeo-Christian beliefs have won America's culture war. Now women have no rights. They are slaves to men and the biblical, patriarchal society in which they live. <em>The Handmaid's Tale</em> is the first-person account of one of these enslaved women.<br><br><strong>Massachusetts Turns Into Saudi Arabia?</strong><br>More than thirty years have passed since <em>The Handmaid's Tale</em> was first published in 1985, but many still think of it as the go-to book for feminist fiction. It makes numerous \"best of\" lists, the kinds with 99 other books everyone should read before dying. Even so, <em>The Handmaid's Tale</em> frustrates me a lot—and not only because it contains run-on sentences and needlessly abandons quotation marks. (This is no train wreck like <a href=\"https://www.goodreads.com/review/show/338987766\" title=\"Seriously, it's so unreadable.\" target=\"_blank\" rel=\"nofollow\">José Saramago's <em>Blindness</em></a>, but it's bad enough.) Simply put, if you can ignore whether you agree or disagree with Margaret Atwood's ideas about politics, religion, and women's rights, the plot and setting make no sense.<br><br>The religiosity of the Reagan era inspired Atwood's dystopia, in which fundamentalist Christians have taken over society. While that premise does give me the heebie-jeebies, Atwood’s taken the idea to a literal extreme to make a point. This ruins the foundation of <em>The Handmaid's Tale</em> because most American fundies would balk at this world. Atwood imagines the extreme <em>of</em> the extreme and in the process completely misunderstands American evangelicalism. <br><br>I'm a heathen bastard and no fan of religion. Fundamentalism has hurt people, particularly women, for millennia. Extremism continues to hurt people every day, especially in some parts of the world, especially in some states. Even so, it's hard to accept Atwood's dystopia when it's set in the U.S., in the near future—and in Massachusetts, one of the most progressive states in the country, one of only <a href=\"http://www.prochoiceamerica.org/what-is-choice/maps-and-charts/map.jsp?mapID=24\" target=\"_blank\" title=\"NARAL: State Constitutional Protection Map\" rel=\"nofollow\">sixteen states</a> in the union with state constitutional protections for abortion (since 1981, I believe). Massachusetts is a liberal bastion when it comes to American women's reproductive rights, so it's an odd setting for this brand of nightmare. In recent decades, <a href=\"http://www.gallup.com/poll/181601/frequent-church-attendance-highest-utah-lowest-vermont.aspx\" title=\"Gallup: Frequent Church Attendance Highest in Utah, Lowest in Vermont\" target=\"_blank\" rel=\"nofollow\">Massachusetts is also one of the least religious states</a>, so it's an odd setting for a theocracy, too.<br><br><a href=\"http://articles.latimes.com/1990-03-04/entertainment/ca-2834_1_atwood-tale-handmaid\" target=\"_blank\" title=\"Los Angeles Times: 'The Handmaid's Tale' - If Puritans Ruled . . . Atwood's Story on Screen\" rel=\"nofollow\">Atwood chose Massachusetts for its puritanical history</a>. I can embrace the connection to the Reagan administration, in the same way I can embrace Orwell's fear of communism in <em>1984</em>, but to imagine an unchanging, puritanical Massachusetts requires a bit too much. <br><br><strong>Societies Don't Change Overnight</strong><br><em>The Handmaid's Tale</em> is told in first person by a woman who’s lived in our present day (more or less), as well as in this dark fundamentalist Tomorrowland. She’s gone from wearing flip-flops and sundresses to a full-body religious habit, color-coded red to match her subservient role. She was married once, had a child. Now she’s another’s property, one of the handmaids sent from one man’s house to another. The hope is that she will become pregnant when a prominent man’s wife cannot. Her life has been flipped and made forfeit. She lives in fear and depression and abuse. This is meant to make me unnerved, and it does.<br><br>But.<br><br>Simply because an author wants to comment on society doesn’t mean he or she can ignore important, logical story elements. The logic part should be emphasized here, I think, given this is supposed to be science fiction, <em>not</em> fantasy. (<a href=\"http://en.wikipedia.org/wiki/The_Handmaid%27s_Tale#Classification_as_science_fiction_or_speculative_fiction\" rel=\"nofollow\">Although Atwood does insist <em>The Handmaid's Tale</em> is speculative fiction</a>, because that further legitimizes her story...or something? Never mind that sci-fi and fantasy are types of speculative fiction.)<br><br>There’s a question I have that never gets answered, not properly at least. <em>How</em> did this happen so quickly? How did we go from \"<a href=\"http://womenshistory.about.com/od/mythsofwomenshistory/a/bra_burning.htm\" target=\"_blank\" title=\"Didn't happen, but still a great phrase, and I have days where I'd love to burn mine.\" rel=\"nofollow\">burning bras</a>\" to having every part of our lives regulated? Why did it take Massachusetts decades, centuries, to reject puritanism, but only a few years(?) to reject liberalism?<br><br>Rights can erode, but you don’t see it happen on such a large scale and so seamlessly, and not overnight. Nothing happens overnight, especially not governmental takeovers in relatively stable, secular societies, which is the book's scenario. <br><br>Societies evolve, one way or another, usually rather slowly. Civil, moral, and regime changes don't sneak up on you. It wasn't the case in <a href=\"http://www.historyplace.com/worldwar2/riseofhitler/begins.htm\" target=\"_blank\" title=\"The History Place: The Rise of Adolph Hitler\" rel=\"nofollow\">Germany before Hitler</a>, in <a href=\"https://www.nytimes.com/books/first/s/spence-mao.html\" target=\"_blank\" title=\"NYT Books: A Child of Hunan\" rel=\"nofollow\">China before Mao</a>, in <a href=\"http://www.bbc.com/news/world-south-asia-12024253\" target=\"_blank\" title=\"BBC: Afghanistan Profile - Timeline\" rel=\"nofollow\">Afghanistan before the Taliban</a>, in <a href=\"http://www.scientificamerican.com/article/climate-change-hastened-the-syrian-war/\" target=\"_blank\" title=\"Scientific American: Climate Change Hastened Syria's Civil War\" rel=\"nofollow\">Syria before its civil war</a>. It's not the case in 2016, with people like Ted Cruz and Donald Trump leading in GOP primary polls. The world may be disappointing and horrible sometimes, but it is rarely surprising.<br><br>If Atwood had built her dystopia on a chain of events that occurred over a longer period of time, or explained how everything unraveled so quickly, I might have been on board with the premise. That isn't how <em>The Handmaid's Tale</em> is written, though. The explanations for the sudden changes are fantastical, at best, dependent on evil, digitized money—be careful with the mobile payments and bitcoins, ladies!—and misogynistic, conservative conspiracies that readers are to believe could bring millions of people to a stupefied halt and change culture in the blink of an eye. <a class=\"jsShowSpoiler spoilerAction\">(view spoiler)</a><span class=\"spoilerContainer\" style=\"display: none\">[Apparently it’s easy to gun down all of Congress while it’s in session. Who knew? <a class=\"jsHideSpoiler spoilerAction\">(hide spoiler)</a>]</span><br><br>I don’t buy it.<br><br>You can change laws all you want, but society, culture, has to be willing to follow the most drastic changes. (This is why the American Drug War has never worked, why prohibition of alcohol never worked, why banning abortion didn't work.) Why was modern American society <em>so</em> willing to enslave women? <br><br>Atwood chucks a plot point at you here or there, hinting at a larger, more complex world through her main character. There’s a vague fertility crisis (of course). There's conflict <em>somewhere</em> between <em>some people</em> about <em>some stuff</em>, but details are never given. Some of this can be excused, what with the limited point of view, but not all. Plot holes aren't mysterious or clever. They're just plot holes.<br><br>By the end of <em>The Handmaid’s Tale</em>, I feel the book is less an exploration of religious extremism and feminism than it is a narrative written for shock value. It’s an irrational feminist’s fears exposed, that the world is out to get you at every turn—especially the men, especially the women controlled and brainwashed by the men. Nowhere is safe. Overall, the summary for this book could be this: Almost anyone with a penis is mostly unfeeling and evil, deep down. (The rest are idiots, I suppose.) He doesn’t care. He will betray you at the first opportunity. Even when you're dead and gone, he will chuckle at your misfortune and demise. No, this isn’t sexist or a generalization. Of course not. Not at all.<br><br>Except it is.<br><br><strong>Asides</strong><br>- For a slightly more accurate portrayal of American Christian fundamentalism and its very awkward relationship with women, see Hillary Jordan's <em>\n  <a href=\"http://amzn.com/1616201932/?tag=fantastikate-20\" rel=\"nofollow\">When She Woke</a>\n</em>. It makes several nods to <em>The Scarlet Letter</em> and <em>The Handmaid's Tale</em>—and better understands its villains and their behavior.<br><br>- Two nonfiction books, Jenny Nordberg's <em>\n  <a href=\"http://amzn.com/0307952509/?tag=fantastikate-20\" target=\"_blank\" rel=\"nofollow\">The Underground Girls of Kabul</a>\n</em> and Ned &amp; Constance Sublette's <em>\n  <a href=\"http://amzn.com/1613748205/?tag=fantastikate-20\" target=\"_blank\" rel=\"nofollow\">The American Slave Coast: The History of the Slave-Breeding Industry</a>\n</em>, will show you what it's <em>really</em> like to live in a society where women are chattel.<br><br>- Some think that because I dislike this book I'm not a feminist, or am a bad feminist. I hate to break it to everyone, but Margaret Atwood is not feminism's god, and <em>The Handmaid's Tale</em> is not a religious text. If I must attach labels to myself, feminist would be one of them, and I'll say and think whatever I damn well please. And as a feminist, I hate how one-dimensional the men are in this book, just as much as I hate how one-dimensional women are in far more books, TV shows, and movies. Deal with it. Or don't. ¯\\_(<U+30C4>)_/¯<br><br><a href=\"http://fantastikate.com/reading/\" title=\"Visit Fantastikate.com for more book reviews.\" rel=\"nofollow\">\n  <img src=\"https://i.gr-assets.com/images/S/compressed.photo.goodreads.com/hostedimages/1380340956i/697219.png\" alt=\"Fantastikate Logo\" class=\"gr-hostedUserImg\">\n</a><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--><!--[\"br\"]--></span>\n  <a data-text-id=\"1606444764647290275\" href=\"#\" onclick=\"swapContent($(this));; return false;\">(less)</a>\n\n          </span>\n        </div>", 
    "<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/20234653?book_show_action=false\">Apr 15, 2008</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Stephanie *Extremely Stable Genius*\" class=\"user\" itemprop=\"url\" name=\"Stephanie *Extremely Stable Genius*\" href=\"/user/show/1085121-stephanie-extremely-stable-genius\">Stephanie *Extremely Stable Genius*</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"it was amazing\"><span size=\"15x15\" class=\"staticStar p10\">it was amazing</span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span><span size=\"15x15\" class=\"staticStar p10\"></span></span>\n\n\n\n        \n\n          <div class=\"uitext greyText bookshelves\">\n            Shelves:\n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=lit\">lit</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=favorites\">favorites</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=2014\">2014</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=read-count-2\">read-count-2</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=re-read\">re-read</a>, \n              <a class=\"actionLinkLite\" href=\"/review/list/1085121-stephanie-extremely-stable-genius?shelf=2017\">2017</a>\n          </div>\n    </div>", 
    "<div class=\"reviewText stacked\">\n              <span id=\"reviewTextContainer20234653\" class=\"readable\">\n            \n<span id=\"freeTextContainer1934983326442817899\" style=\"display: none;\">5/22/19<br><br>Looking back on my original review, it reads as quaint compared to the draconian state laws recently being passed, my state of Ohio being one of them. Make no mistake, this not about ‘life’ it’s about controlling women. If you can’t decide what happens to your own body you do not have freedom. This is about bodily autonomy. <br><br>Women have the RIGHT to legal and safe abortions with no qualifications. The fact that the narrative has gone to ‘in cases of rape and incest’ is troubling. Rape...inc</span>\n  <span id=\"freeText1934983326442817899\" style=\"\">5/22/19<br><br>Looking back on my original review, it reads as quaint compared to the draconian state laws recently being passed, my state of Ohio being one of them. Make no mistake, this not about ‘life’ it’s about controlling women. If you can’t decide what happens to your own body you do not have freedom. This is about bodily autonomy. <br><br>Women have the RIGHT to legal and safe abortions with no qualifications. The fact that the narrative has gone to ‘in cases of rape and incest’ is troubling. Rape...incest...life of the mother....horrible birth defects....you’re young, single and not ready....you have five kids and can’t afford more, it doesn’t matter! <br><br>This is a medical procedure and pregnancy is a risky condition, it can cause death. Every woman has the right to decide whether or not they want to take that risk. Period.<br><br>Men don’t have anything that compares to this. No law is forcing them have vasectomies, or even denying them their bonner pills.<br><br>7/7/17 I'm just going to leave this here.... ■■■■ Paul Ryan.... but not literally, ew. <br><br><a href=\"http://nytlive.nytimes.com/womenintheworld/2017/07/06/house-speaker-paul-ryan-reportedly-implements-new-no-sleeveless-dress-code-for-women/\" rel=\"nofollow\">Sleeveless women? My stars and garters!</a><br><br>03/31/17. So, this Russia thing.... Am I right?<br><br>2/5/17.....just another giant step towards making this book a <a href=\"http://dailysignal.com/2017/02/03/religious-freedom-advocates-urge-trump-to-sign-executive-order/\" rel=\"nofollow\">reality</a>, like they always dreamed of.<br><br>Original review written in 2o12:<br><br>WARNING: This review is being written after I worked a 13 hour day, with another one on the horizon tomorrow, and a glass of wine and while watching the Rachel Maddow show. Current events have put this book on the forefront of my mind, and damn it I got to get this out. <br><br>I have never written a review on The Handmaid's Tale because I love the book, and it is so hard to write about a book you love. <br><br>Ehh, what the hell.<br><br>OfFred was a normal everyday woman with a career, a name, a life like all women have come to expect and take for granted in this age. When the Religious Right came into power, they began to put into practice their insane beliefs which strip women of their identity, their rights, their body, their very name. Women are to be called Of(whatever asshat they belong to), instead of, say Beatrix. Reproduction is an issue because all the toxins in the environment have rendered many women infertile. But if you are fertile, woe to you, you get to be a baby factory against your will, get promised to some jerk you don’t love or even like because someone deemed him important enough to breed. Oh, come on!<br><br>This book was written in 1986, FYI. I thought it was scary and sort of possible when I first read it, but farfetched. This could NEVER happen in the United States of America. Never would it be allowed to happen here, we are too educated.<br><br><br>So………<br><br><br>I turn on the news (in twothousandandfrikntwelve) and certain religious factions on the right are trying to defund Planned Parenthood, because they perform abortions which is only 3% of what they do (with NO federal $ going towards them). Mostly PP provides healthcare to women who wouldn't get it otherwise………..icky poor women. <br><br>Now it’s birth control? Seriously? Birth control??????? Did I wake up in 1950? Am I stuck in a Atwood novel? 98% of Catholic women (technically I’m one of them) use/used birth control. Even they are asking WTF? <br><br>I’m not sure what these people are trying to do. There are more women than men and we vote……unless that’s the next right on the chopping block.<br><br>------------------------------------------------------------<br><br>There have several updates to this review that I have removed to make room for the next. what follows is the most recent one. <br><br>------------------------------------------------------------<br>It’s been nearly a week since the unimaginable happened and I had to let the shock wear off before I could put a coherent, non-rage filled update on this review. Not that I don’t have rage, I have plenty to spare, but I think it’s now at a level that is manageable enough for me not to just type out a string of obscenities. That being said… <br><br>Update 11/14/16: An unqualified, racist, xenophobic, sexist, pathological liar, psychopathic reality star was elected to be the 45th president of the United States and the leader of the free world. <br><br>■■■■!<br><br>The United States has officially shat the bed. Few foresaw it, but in hindsight, it was coming down the road for a very long time. The <a href=\"https://youtu.be/hI8TCA3fJcs\" rel=\"nofollow\">United (divided) States</a> voted for Hillary Clinton on whole (popular vote) by over 2,000,000 votes and counting (millions are still out in California, for example), yet Donald Trump is our president elect (gag) due to an antiquated electoral college system (which I could explain, but I’m not because Google can do that better than I can.) Now, I’m all for ditching the electoral college, unless the electors decide to do what it was intended to do under this circumstance; to save us from ourselves. See, our founding fathers knew that we would fall for some con artist, demagogue at some point in the future, so they wisely created the electoral college, a group of actual human beings trusted upon to stop such a calamity. I implore the folks of current electoral college recognize this election as a collective loss of sanity of less than a quarter of the population of this nation, and on December 19th put their votes towards the popular vote winner, Hillary Clinton. I realize that this is unlikely, but one can dream.<br><br>How did this happen? There are many factors involved. Lots of opportunity for pointing fingers and fighting amongst ourselves, which I will admit to being a party to…..guilty. But, in my opinion, what it boils down to is these four things: Division, misinformation, apathy and fear of the ‘other’. <br><br>Division: We are all in our own comfortable bubbles, digesting the information we are most comfortable with. For example, I never believed there was this much hate it this country because I didn't want to look at it; I knew it was there of course, but not at the level that it appears to be. Everyone wants to live where they feel they belong. Amongst those that are like minded and reaffirm your very rightness. Liberals don’t want to live in Indiana (or Ohio….sigh) any more than a conservative want’s to live in Washington state. We even do this in our social media as well (guilty again). This is what messed us up with the electoral college. <br><br>Misinformation: I am not going to tell you who’s right or wrong here, I’ll let <a href=\"http://www.businessinsider.com/study-watching-fox-news-makes-you-less-informed-than-watching-no-news-at-all-2012-5\" rel=\"nofollow\">this study</a> speak for itself.<br><br>Apathy: Half…HALF… the country didn’t vote. You guys suck.<br><br>Fear of the other: This country harbors more racism than I can comprehend. The white people in this country seemed a little angry about the black man in the white house and the white men were staunchly determined not to have a woman (white or not) follow him. I don’t mean all white men, just too many of them (chill.) “The advantage for Trump among men is larger than the 7-point advantage Romney had in 2012 and much different than in 2008, when men preferred Obama over McCain by a single point.”-<a href=\"http://www.pewresearch.org/fact-tank/2016/11/09/behind-trumps-victory-divisions-by-race-gender-education/\" rel=\"nofollow\">PewResearchCenter</a>. But then there are the white women, 53% went for Trump…..oh my sisters, I have no words.<br><br>Which brings me to the reason why this update is relevant to this review and to this book (for those who tell me that my opinion is unwarranted....again.) Is the United States a more racist country, or a more sexist country? America has spoken, at least the ones who cared to speak, and the answer is “a goodly amount of both”, but in this election sexism won and women lost.</span>\n  <a data-text-id=\"1934983326442817899\" href=\"#\" onclick=\"swapContent($(this));; return false;\">(less)</a>\n\n          </span>\n        </div>", 
    "<div class=\"reviewHeader uitext stacked\">\n        <a class=\"reviewDate createdAt right\" href=\"/review/show/78979781?book_show_action=false\">Nov 25, 2009</a>\n\n      <span itemprop=\"author\" itemscope=\"\" itemtype=\"http://schema.org/Person\">\n        <a title=\"Pollopicu\" class=\"user\" itemprop=\"url\" name=\"Pollopicu\" href=\"/user/show/2345961-pollopicu\">Pollopicu</a>\n      </span>\n\n        rated it\n        <span class=\" staticStars notranslate\" title=\"did not like it\"><span size=\"15x15\" class=\"staticStar p10\">did not like it</span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span><span size=\"15x15\" class=\"staticStar p0\"></span></span>\n\n\n\n        \n\n    </div>")

I had to shorten it again, but if it is too short I could perhaps send you the entire output in private message if that might help?

with the reviews.html that you gave me, and the lapply() functional call that we discussed, do you get your original error or not ?
why do you want to change reviews.html now? please give a reason.

I'm sorry if I misunderstood!

No, I get the following error after I changed the lapply line:

Error in `$<-.data.frame`(`*tmp*`, "review", value = c("One of these books that make you think: \"wow, you can write like that???\". Very inspiring. It's a collection of snippets from a weird, neo-futurist city where cyberfish coexist with electronymphs and robot keepers of the wind, and where the human is judged by trees. A hypnotising piece of good poetry.",  : 
  replacement has 4 rows, data has 5

I'll add a screenshot, maybe that explains it better?


I don't really want to change reviews.html, but because it's such a popular book the reviews keep changing and they were different when I ran the script again, so that's why the output I sent you in my last message was different than the previous one. Is this what you meant? (I'm sorry if I got it wrong, English is not my native language, so sometimes I misunderstand)

you get an error many many lines beyond the error we were originally focussed on.
Are we to conclude that we somehow fixed that original error ?

or perhaps the particulars of the reviews.html that you shared with me simply did not produce examples of that error, so it was a less than ideal example to have shared ?

@nirgrahamuk Hello again, I'm sorry I couldn't reply anymore last night. I'm not sure the original error has been fixed (it appeared at the same place as this one), I think the new error may just be kind of "covering" it, if that makes sense?
That might be the case, because I had to shorten it so much. I will send you a longer sample via private message, if that is alright with you?

ok, if you have to transfer slightly larger objects.
best save them out to binary RDS objects by using saveRDS.
You could then use a hosting service such as onedrive, dropbox, googledrive, github etc etc. to share your file through a link.
You can write such a link in a forum post.