R Studio cloud Getting crashed repeatedly

My project id Is https://rstudio.cloud/project/224456 . It is public currently .

This is getting crashed everytime I am trying to execute this following chunk .



library(knitr); library(dplyr); library(doParallel); library(tm); library(SnowballC)
library(stringi); library(tm); library(ggplot2); library(wordcloud)

path1 <- "/cloud/project/final/en_US/en_US.blogs.txt"
path2 <- "/cloud/project/final/en_US/en_US.news.txt"
path3 <- "/cloud/project/final/en_US/en_US.twitter.txt"

conn <- file(path1, open="rb")
blogs <- readLines(conn, encoding="UTF-8"); close(conn)
# Read news data in binary mode
conn <- file(path2, open="rb")
news <- readLines(conn, encoding="UTF-8"); close(conn)
# Read twitter data in binary mode
conn <- file(path3, open="rb")
twitter <- readLines(conn, encoding="UTF-8"); close(conn)
# Remove temporary variable
rm(conn)

WPL <- sapply(list(blogs,news,twitter),function(x) summary(stri_count_words(x))[c('Min.','Mean','Max.')])
rownames(WPL) <- c('WPL_Min','WPL_Mean','WPL_Max')
stats <- data.frame(
  FileName=c("en_US.blogs","en_US.news","en_US.twitter"),      
  t(rbind(
    sapply(list(blogs,news,twitter),stri_stats_general)[c('Lines','Chars'),],
    Words=sapply(list(blogs,news,twitter),stri_stats_latex)['Words',],
    WPL)
  ))
head(stats)

Please help me .

It looks like the combined size of the text files in the folder you're looking at is over 500 Mb.

Given that the packages that you're loading are also fairly large, it's possible you're running into a memory issue.

It looks like the files are slightly larger when in-memory in the R session. So, for me at least, things stopped working when I try to read in the twitter data:

How far can you get through this script before crashing if you're going through step by step?

1 Like

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.