Looping over multiple datasets

General
,
1

I have written a code that which gets all of the .dat files in the subdirectory that I am storing my data files. But now the next step is to:

  1. Write a chunk of code that loops over each file in the list, reads it in, extracts the 10 Hz U and V wind data, and computes the 30-minute mean U and V wind velocities.
    Each dataset is given in 10 Hz, therefore, I only have to pull the U (Ux) and V (Uy) wind data out and then calculate the 30-minute mean of the two.

Here is my code thus far:
pth <- "C:/Users/crazy/OneDrive/Documents/Micrometeorology Fall 2022/Data Unzipped"
dat.files <- list.files(path = pth,
recursive=T,
pattern= "dat",
full.names = T)
print(dat.files)
##This code above does show all 5 datasets that I have to work with
##Now I have to loop over each file in the list, read it in, extract the U and V wind data, and compute the 30-minute mean U and V wind velocities
for(i in 1:length(dat.files)) {
fni <- file.path(pth,dat.files[i])
read_csv(dat.files)
}
This code above where I try to loop is working so far, but I'm not sure how to pull out the U and V data from each file nor how to calculate the 30-minute mean of the U and V again from each dataset; there are 5 datasets in total. I'm fairly new to looping and R-coding in general.

ANY AND ALL HELP IS DEEPLY APPRECIATED!

2

Hi, can you provide a reproducible example of what one of the datasets look like?

Also, using the purrr or apply functions will be faster than a for loop.

3

Hey williaml,

Here is a picture of what the dataset looks like:

Code
Code1133×606 41.7 KB

How could I use the purrr or apply function to do this loop faster? These datasets each have 864,000 columns and there are 5 datasets in total.

4

Hi, can you provide a reproducible example (not a screenshot)? e.g. paste the output of dput(head(df)) where df is your dataset.

Also which columns are the wind velocities that you want to compute?

Do you only want the means? Do you want to keep the other columns?

-- edit --
you mentioned it in the first post.

U (Ux) and V (Uy) wind data out and then calculate the 30-minute mean of the two