Issues about the loop in Rstudio

[nchan08]

I have more than 1000 csv files. I would like to combine in a single file, after running some processes. So, I used loop function as follow:

setwd("C:/....") files <- dir(".", pattern = ".csv$") # Get the names of the all csv files in the current directory.

for (i in 1:length(files)) { obj_name <- files %>% str_sub(end = -5) assign(obj_name[i], read_csv(files[i])) }

Until here, it works well.

I tried to concatenate the imported files into a list to manipulate them at once as follow:

command <- paste0("RawList <- list(", paste(obj_name, collapse = ","), ")") eval(parse(text = command))

rm(i, obj_name, command, list = ls(pattern = "^g20")) Ref_com_list = list()

Until here, it still okay. But ...

for (i in 1:length(RawList)) { df <- RawList[[i]] %>% pivot_longer(cols = -A, names_to = "B", values_to = "C") %>% mutate(time_sec = paste(YMD[i], B) %>% ymd_hms())%>% mutate(minute = format(as.POSIXct(B,format="%H:%M:%S"),"%M"))

...(some calculation) Ref_com_list [[i]] <- file_all }

Ref_com_all <- do.call(rbind,Ref_com_list)

At that time, I got the error as follow:

Error: Can't combine A and B . Run rlang::last_error() to see where the error occurred.

If I run individual file, it work well. But if I run in for loop, the error showed up.

Does anyone could tell me what the problem is?

Thanks a lot in advance.

[pieterjanvc]

Hi,

Do all of the csv files have the same columns? In other words, can they be just pasted together one below the other? If not, please show an example of what needs to happen to the files first before they can be merged. That pivot_longer seems something you could also do after the 1000 files have been merged it seems now, but I'm not sure.

Here is an example on how to do it for data files that have the same columns

#Create data
for(i in 1:3){
  write.csv(
    data.frame(id = i, x = runif(5)),
    paste0("file", i, ".csv"), row.names = F
  )
}


#Get all filenames
filePaths = list.files(".", "file\\d.csv", full.names = T)

#Read all files into a list
myData = lapply(filePaths, read.csv)

#Merge the list
myData = do.call(rbind, myData)
myData
#>    id          x
#> 1   1 0.67335505
#> 2   1 0.88818906
#> 3   1 0.09722986
#> 4   1 0.08021714
#> 5   1 0.40849455
#> 6   2 0.66050555
#> 7   2 0.14155156
#> 8   2 0.19359541
#> 9   2 0.52252941
#> 10  2 0.58511143
#> 11  3 0.66896675
#> 12  3 0.21849641
#> 13  3 0.86556275
#> 14  3 0.65303338
#> 15  3 0.49257413

^{Created on 2022-03-02 by the reprex package (v2.0.1)}

PJ

[nchan08]

Thank you, PJ!
Now I found out the reason why it happened. There was another file which is not the same file name but with the same file type. So, the code read all the files, and provided the error.
I am sorry I made you all confused.
Thank you so much!