Hello RStudio Community,
I am working with some very large data coming from ArcGIS as .csv files. The largest dataset is approx. 60 columns and 70 million rows. But even with somewhat smaller files (60 cols and 30 million rows), RStudio crashes with no warning while trying to read it in. Is there a way to fix this?
Perhaps there is a less computationally-demanding method to import originally-shapefile, turned .csv files to R? The smallest .csv dataset that I have is 60 cols and 11 million rows and it imports successfully in 10 minutes, then I save it as RDS, and importing it from RDS is much faster (<1min). Can I convert my large files to an easier file type beforehand?
Thank you in advance for any advice.