Hello there,
I have 5,3GB of data, 35.360 zip files with 1 csv file inside each of them, all organized inside 41 folders, those are log files. The file names are organized like this:
Folder 2018-10-25:
2018-10-25-00-00-0e41.csv.gz;
2018-10-25-00-00-7f7d.csv.gz;
2018-10-25-00-00-32fa.csv.gz and so on;
Folder 2018-10-26:
2018-10-26-00-00-0e41.csv.gz;
2018-10-26-00-00-7f7d.csv.gz;
2018-10-26-00-00-32fa.csv.gz and so on.
Last folder 2018-12-04.
How can I read all those files in R as just one file? Any tips to work with such a great amount of data?
Kind Regards,
Luiz.