I have a huge dataset in STATA that i want to read into R. This dataset has around 7 million observation and can't be read into R. error message appear:
Error: cannot allocate vector of size 27.8 Mb
How can I read such a big file into R studio?
Which code did you run for attempting to import the file?
I am using read.dta13().. I managed to increaase the memory limit by using memory.limit(size=500000). But still it is so slow to perform any operation in Rstudio on these huge data. Is it hardware problem, or i can do something about it ?
Which package does the read.dta13() function belong to?
Also, working with very large datasets in memory can be slow generally. But there are packages that can help you with that:
-
data.table: https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.html -
sparklyr: https://sparkfromr.com/ -
disk.frame: https://diskframe.com/
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.
If you have a query related to it or one of the replies, start a new topic and refer back with a link.