i've this code where it takes the data from a database and since each six rows i've got informations about a single unit, i've used the function split to group in each element of a list, in tibbles, these infos. After this i've done in such a way that if it is found the string "#ERROR", in one of the tibble it will delete the element of the list (the tibble of 6 rows
library(tidyverse) dataset <- read_excel('test.xlsx') my.list<-split(dataset, rep(1:119, each = 6)) new.list <- my.list[sapply(1:length(my.list), function(i) all(my.list[[i]] != "#ERROR"))]
This code is working but in the moment in which i use another database (the real one, i've been using this to be faster in trials) which would result in a
my.list which is split in more than 17k elements, i think creates problems with the
sapply function as it gives back
new.list which have reduced the number of rows but they are empty. I've already checked possible errors in the different formattation in the database and that's not the problem. how differently can i write this to make it works?
I've also thought it could have been a problem of memory as my.list is already almost 2 gigas heavy, i've got 8 gigas of RAM and an i7 processor