Automatically export graphs for all the time series in a given data.frame, column by column, every X data points

General
,
1

Dear all,

I would very much appreciate your help.

Thank you very much and I hope you can help me as I am not really able to do anything like that by myself. If I am asking something that is not appropriate, I am really sorry and I do apologise in advance.

I need to visually inspect a lot of graphs (thousands) for two time series. One of the two is fixed (let's name it GW) and will not change, whilst the others (let's name them Pz1, Pz2, Pz3,...Pz n) will be different for each graph: basically, I need to visually compare GW with all the possible Pz, given a fixed-length (i.e., 10 data points).

The data are in a frame format, i.e.,:

    structure(list(GW = c(0.08, 0.04, 0.35, 0.54, 0.39, 0.94, 0.51, 
    0.01, 0.44, 0.63, 0.14, 0.79, 0.43, 0.73, NA, NA, NA, NA, NA, 
    NA, NA, NA, NA), Pz1 = c(-2.459826614, -2.905007821, -2.241113224, 
    -1.549264338, -1.761962438, -1.282612221, -0.428828702, 1.659042479, 
    2.63518648, 3.076022461, 1.886216859, 0.124473561, -1.025720789, 
    -1.969461882, -2.216075605, -2.508858567, -2.495831241, -2.36513321, 
    -2.295002655, -2.103511241, -1.009051105, -0.619616011, -0.941517493
    ), Pz2 = c(-2.916090168, -3.262459435, -2.455396094, -1.488106654, 
    -0.417171756, -1.781014095, -0.605012986, 1.037062685, 1.977265974, 
    2.587846362, 2.499228916, 1.0852274, -0.503736287, -1.829562138, 
    -2.410220303, -2.135932209, -2.105646454, -2.79131286, -2.446274505, 
    -2.412041743, -1.467408853, -1.207262381, -1.286558601), Pz3 = c(-2.507944967, 
    -3.718722989, -2.812847708, -1.702389524, -0.356014073, -0.436223413, 
    -1.10341486, 0.860878402, 1.355286181, 1.929925855, 2.011052817, 
    1.698239458, 0.457017552, -1.307577636, -2.270320559, -2.330076907, 
    -1.732720095, -2.401128073, -2.872454155, -2.563313593, -1.775939355, 
    -1.665620129, -1.874204971), Pz4 = c(-2.526729696, -3.310577787, 
    -3.269111262, -2.059841138, -0.570296943, -0.375065729, 0.241375822, 
    0.362476528, 1.179101897, 1.307946062, 1.35313231, 1.210063358, 
    1.07002961, -0.346823797, -1.748336058, -2.190177163, -1.926864794, 
    -2.028201714, -2.482269368, -2.989493243, -1.927211205, -1.974150631, 
    -2.332562719), Pz5 = c(-3.284551238, -3.329362517, -2.86096606, 
    -2.516104692, -0.927748557, -0.5893486, 0.302533506, 1.70726721, 
    0.680700023, 1.131761778, 0.731152517, 0.552142851, 0.58185351, 
    0.266188261, -0.787582218, -1.668192661, -1.78696505, -2.222346412, 
    -2.109343009, -2.599308456, -2.353390855, -2.125422481, -2.641093221
    ), Pz6 = c(-4.011896321, -4.087184059, -2.87975079, -2.107959491, 
    -1.38401211, -0.946800213, 0.088250636, 1.768424893, 2.025490705, 
    0.633359905, 0.554968233, -0.069836942, -0.076066996, -0.221987839, 
    -0.174570161, -0.707438822, -1.264980548, -2.082446669, -2.303487708, 
    -2.226382097, -1.963206068, -2.551602131, -2.792365071), Pz7 = c(-4.769878994, 
    -4.814529142, -3.637572331, -2.12674422, -0.975866909, -1.403063767, 
    -0.269200978, 1.554142023, 2.086648388, 1.978150587, 0.05656636, 
    -0.246021225, -0.698046789, -0.879908346, -0.66274626, -0.094426764, 
    -0.304226709, -1.560462167, -2.163587964, -2.420526796, -1.59027971, 
    -2.161417344, -3.218544722)), class = "data.frame", row.names = c(NA, 
    -23L))

Basically, I am looking for something that can automatise the entire process and save all the graphs, individually, in a folder so I can patiently check them graphically.

On top of that, both A and B..., should be normalised (Z-Score) before being plotted (i.e., Z=x-mean/Standard deviation), should include some kind of coordinates so I can find them in the matrix (i.e., Graph #4=B5-B8, and ideally (it would be a dream) have also spearman correlation coefficient and RMSE values (please see the image below).

image
image1253×789 96.4 KB

Unfortunately, I have no idea where to start from and your help would be immensely appreciated.

I am sorry if it is confusing and if you need further clarification please, do not hesitate to let me know and I will try to explain it better.

Thank you very much!

2

Not at all!

See this recent thread. Basically, the approach is to prepare a code snippet to do a single plot, convert that code into a function and then map that function to list Pz1 ... Pz_n as arguments to the function.

Can you repost with a reprex? See the FAQ: How to do a minimal reproducible example reprex for beginners. We can help composing the function from the example, if needed, and then the mapping function.

1 Like
3

Dear @technocrat thanks a lot for your help and for your time!

I tried to understand that thread, but I got completely lost.

my data (part of it) are those posted above:

I am sorry I can't provide more info in "R-appropriate format", but I don' really know where to start from (in terms of writing a code for this purpose).

I created an Excel spreadsheet (link below) that (I hope) explains what I am after. Would you be so kind to check it out?

Please, let me know if it is not clear and I will try to address any questions!

Thank you very much indeed!

4

The script below is a partial solution. It does not address how to generate the individual plots (instead it uses a simplified line plot). You will need to provide the code to do that (post if help is required) so that the make_plot function can be modified. What the script does, however, is to loop through the Pz1 ... Pz7 columns, create a plot object and save a pdf.

All of the annotations that are needed can be done similarly and assembled into each plot as a text annotation. Finally, if desired, plots can be gang printed in 3x3 sets.

Another approach to consider, that may be easier to review, would embed a sparkline-style plot within a table with the RMSE, etc. values.

image
image1608×1546 248 KB

# DATA

dat <- data.frame(GW = c(
  0.08, 0.04, 0.35, 0.54, 0.39, 0.94, 0.51,
  0.01, 0.44, 0.63, 0.14, 0.79, 0.43, 0.73, NA, NA, NA, NA, NA,
  NA, NA, NA, NA
), Pz1 = c(
  -2.459826614, -2.905007821, -2.241113224,
  -1.549264338, -1.761962438, -1.282612221, -0.428828702, 1.659042479,
  2.63518648, 3.076022461, 1.886216859, 0.124473561, -1.025720789,
  -1.969461882, -2.216075605, -2.508858567, -2.495831241, -2.36513321,
  -2.295002655, -2.103511241, -1.009051105, -0.619616011, -0.941517493
), Pz2 = c(
  -2.916090168, -3.262459435, -2.455396094, -1.488106654,
  -0.417171756, -1.781014095, -0.605012986, 1.037062685, 1.977265974,
  2.587846362, 2.499228916, 1.0852274, -0.503736287, -1.829562138,
  -2.410220303, -2.135932209, -2.105646454, -2.79131286, -2.446274505,
  -2.412041743, -1.467408853, -1.207262381, -1.286558601
), Pz3 = c(
  -2.507944967,
  -3.718722989, -2.812847708, -1.702389524, -0.356014073, -0.436223413,
  -1.10341486, 0.860878402, 1.355286181, 1.929925855, 2.011052817,
  1.698239458, 0.457017552, -1.307577636, -2.270320559, -2.330076907,
  -1.732720095, -2.401128073, -2.872454155, -2.563313593, -1.775939355,
  -1.665620129, -1.874204971
), Pz4 = c(
  -2.526729696, -3.310577787,
  -3.269111262, -2.059841138, -0.570296943, -0.375065729, 0.241375822,
  0.362476528, 1.179101897, 1.307946062, 1.35313231, 1.210063358,
  1.07002961, -0.346823797, -1.748336058, -2.190177163, -1.926864794,
  -2.028201714, -2.482269368, -2.989493243, -1.927211205, -1.974150631,
  -2.332562719
), Pz5 = c(
  -3.284551238, -3.329362517, -2.86096606,
  -2.516104692, -0.927748557, -0.5893486, 0.302533506, 1.70726721,
  0.680700023, 1.131761778, 0.731152517, 0.552142851, 0.58185351,
  0.266188261, -0.787582218, -1.668192661, -1.78696505, -2.222346412,
  -2.109343009, -2.599308456, -2.353390855, -2.125422481, -2.641093221
), Pz6 = c(
  -4.011896321, -4.087184059, -2.87975079, -2.107959491,
  -1.38401211, -0.946800213, 0.088250636, 1.768424893, 2.025490705,
  0.633359905, 0.554968233, -0.069836942, -0.076066996, -0.221987839,
  -0.174570161, -0.707438822, -1.264980548, -2.082446669, -2.303487708,
  -2.226382097, -1.963206068, -2.551602131, -2.792365071
), Pz7 = c(
  -4.769878994,
  -4.814529142, -3.637572331, -2.12674422, -0.975866909, -1.403063767,
  -0.269200978, 1.554142023, 2.086648388, 1.978150587, 0.05656636,
  -0.246021225, -0.698046789, -0.879908346, -0.66274626, -0.094426764,
  -0.304226709, -1.560462167, -2.163587964, -2.420526796, -1.59027971,
  -2.161417344, -3.218544722
))

# PREPROCESSING
# create an index variable to use as the x-axis and rearrange the data
# frame so that it and the GW variable occur last

dat["obs"] <- 1:nrow(dat)
head(dat)
dat <- dat[,c(2:ncol(dat),1)]
head(dat)

# FUNCTIONS
# make a plot by column number and save it
make_plot <- function(x) {
  require(ggplot2)
  base_plot + geom_line(aes(obs,GW)) +
    geom_line(aes(obs,dat[,x])) + 
    theme_minimal()
    plot_name = here(paste0("figures/",var_names[x],".pdf"))
    ggsave(device = NULL, filename = plot_name)
    }

# HELPER OBJECTS
# base plot

base_plot <- ggplot(dat,aes(obs,GW)) + geom_line()
base_plot

# indices
cols_to_plot <- 1:(ncol(dat)-2)
var_names <- names(dat[1:7])

# MAIN

for (i in seq_along(cols_to_plot)) make_plot(i)

dir("figures/")
[1] "Pz1.pdf" "Pz2.pdf" "Pz3.pdf" "Pz4.pdf" "Pz5.pdf" "Pz6.pdf" "Pz7.pdf"

Example figure:

Pz7.pdf (4.5 KB)

1 Like
5

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.