How to upload multiple text files (~10.000) in R ?

[Michi]

I have multiple text files to Upload to RStudio. Based on my research I have not found a a way to import all files together.
All files are saved separately.

Normally I am taking the import dataset, but with these way I only got the chance to upload one text file.
Later I want to summarize the data as one data frame.

Thanks

[FJCC]

Here is a simple way to make a list with each element being the content of one file and then making a single data frame with all of the data. I assumed the files are in the current working directory and that the files have a similar structure to each other.

library(dplyr,warn.conflicts = FALSE)
FILE_NAMES <- list.files(pattern = ".csv")
AllFiles <- vector(mode = "list", length = length(FILES))
names(AllFiles) <- FILE_NAMES 
for (NM in FILE_NAMES) {
  AllFiles[[NM]] <- read.csv(NM)
}
AllInOne <- bind_rows(AllFiles)

[andresrcs]

A tidyverce based solution would be like this:

library(tidyverse)

list_of_files <- list.files(path = "path/to/your/files",
                            pattern = "\\.csv$",
                            full.names = TRUE)
AllInOne <- list_of_files %>%
    map_dfr(read.csv)

[Michi]

First I want to say thanks.

I actually tried your way.

But the values (list_of_files) character is still empty.
If I continue to summarize the data (AllInOne) keeps still zero observations of 0 variables.

library(tidyverse)

list_of_files <- list.files(path = "path/to/my/files",
pattern = "\.csv$",
full.names = TRUE)
AllInOne <- list_of_files %>%
map_dfr(read.csv)

I might think that the problem is because my data is a not a .csv data.
I'm using a Mac and the data is just shown in the text edit.

It would be great if you could have a look at it.

[Michi]

thanks so much.

But actually it is not working.

I'm not sure if the problem is that my data is not in .csv.

My Mac tells me the data files are just documents and I'm only able to open them via TextEdit.

Is there a option to upload the data if they are not .csv?

Thanks in advance

[andresrcs]

You haven't tell us much about your data or provided a sample so we can only give you general examples that you have to adapt to your needs.

If you need more specific help, please provide a proper REPRoducible EXample (reprex) illustrating your issue.

[Michi]

51060.pdf (101.2 KB)
Here is one of my data sets.
I transferred it to pdf.
My target is to get a structure to all data sets after I uploaded into ones.

Maybe you can help me to find out a code to integrate these 10.000 data sets into one.
Would be perfect, thanks in advance

Have a nice start in the year 2021.

[FJCC]

The link you posted leads to an essay which seems to contain no numeric data. Is that what you intended? Do you want to do text analysis? I thought you had numeric data in the txt file format.

[Michi]

All kind of data are texts or comments like the 51060.pdf I have posted before.

First I want to order them, later on I prefer to text analysis.

Sorry for the misunderstanding.

But to do all these tools I first have to implement all kind of texts in R.

[antdurrant]

It seems like readtext::readtext() will be your friend here. It can handle most common text formats and I have found it to be extremely clever.

readtext function | R Documentation

If you don't have it installed already, you will need to install.packages("readtext") first.

library(readtext)
readtext("./path/to/directory/.")

The dot at the end will collect all the files in the directory into a dataframe with:

• doc_id: the filename
• text: the text in the document

From there, you can clean/manipulate as needed.