It looks like my first messages failed.
I have 2 initial questions:
Which RStudio product is best suited to handle large datasets? I do not know the actual file size but I do know the table contains 13 columns and several million rows.
I am currently working with the free (cloud) version of RStudio. I have figured out that I can not import a file into the RStudio workspace without uploading the file first. I have stepped through the upload process but have yet to complete a successful upload. I am sure that I am missing something simple. Does anyone have any suggestions on how to successfully upload a CSV file?
Thank you in advance.
Hi, good to see you back.
I have no idea about Question 2. I have never worked in the Cloud.
As for Question 1, I think the answer depends very much on what you are doing. It depends on the available RAM, I believe.
Just having 13 columns and several million rows on my small laptop is not necessarily a problem. I just created a data.table of 54 million random numbers in 13 columns It is 944.3 MB in .csv format.
A couple of simple calculations work well.
library(data.table) xx <- data.table(matrix(rnorm(52000000), ncol = 13)) mean(xx$V1) lm(V1 ~ V2, data = xx)
So you may not have any great problem in R with that size of data set. I have read that you may have storage problems in though Cloud but I don't know what those limitations are.
If you have a really large dataset that exceeds R's memory limits, the usual way is to store the data in a database and pull in the data as needed. R has packages that seem to allow one to connect to most available database programs. See Data Analysis and Visualisation in R for Ecologists for a discussion.
A relatively new one that seems to be gaining popularity is duckdb
I have been trying to use write.csv() to export a CSV file my local (mac) computer. So far, I have been unsuccessful. ( I am still using cloud version of RStudio)
When I try and establish a working directory on my computer, I get the following error:
Error in setwd("C:/Users") : cannot change working directory
I am able to import data from my computer into RStudio so I am hopeful that I can export a file from RStudio back to my local computer.
I apologize for messaging you directly but I am not able to send off a new message for some reason.
Wondering if you have any ideas on this?
I have seen that before and I am trying to remember what I did. I am also hampered by the fact have not used Windows close to 15-16 years.
Have a look at How To Fix R Error cannot change working directory
The suggestion about using
I am not able to send off a new message for some reason.
That is really weird if you can send me a message. Are you getting any error messages?
I do not know of any way to change the working directory in RStudio Cloud to a location on your local computer.
If you want to download a file to your computer, click on the Files tab and then More (with the blue gear icon). Select Export... from the drop down menu.
Are you on a Mac? Do Mac's have a "c" drive?
Thanks again !! I will take a look at the link.