Calendar Heatmaps

Hi,

I am trying to create a time-based heatmap similar to this:

I have successfully created this one:

This is simply the sum of the counts for each year.

But, I need to put the counts for each month for each year (even if counts are 0 for a month) and want to have "Year" labels placed across the bottom ("Month" labels not neccessary). I think I have to create a new column with Month-Date, but not sure how to label this correctly???

My code (having problem with plyr/dplyr):

detach("package:plyr", unload = TRUE) # then...
#> Error in detach("package:plyr", unload = TRUE): invalid 'name' argument
wellSummary3 <- mass %>%
  group_by(year, month, well) %>%
  summarise(N = n())
#> Error in mass %>% group_by(year, month, well) %>% summarise(N = n()): could not find function "%>%"

#overall summary
plot5 <- ggplot(wellSummary3, aes(x = year, y = well)) + geom_tile(aes(fill = N),colour = "white") +
  scale_fill_gradient(low = col1, high = col2) +  
  guides(fill=guide_legend(title="Measurements")) +
  labs(title = "Water Level Measurments Per Month",
       x = "Year", y = "Number of Events") +
  theme_bw() + theme_minimal() + 
  theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())
#> Error in ggplot(wellSummary3, aes(x = year, y = well)): could not find function "ggplot"
plot5
#> Error in eval(expr, envir, enclos): object 'plot5' not found

Created on 2021-07-29 by the reprex package (v1.0.0)

A subset of my data:

mass<- tibble::tribble(
            ~well,        ~date, ~year, ~month, ~result,
         "DEPFLD", "11/13/2017", 2017L,  "Nov",      37,
           "MWBS", "11/13/2017", 2017L,  "Nov",   22.15,
         "DEPPBD", "11/14/2017", 2017L,  "Nov",    19.5,
         "DEPPBS", "11/14/2017", 2017L,  "Nov",    17.5,
           "MW01", "11/14/2017", 2017L,  "Nov",     4.3,
           "MW02", "11/14/2017", 2017L,  "Nov",    15.7,
           "MW04", "11/14/2017", 2017L,  "Nov",      38,
           "MW06", "11/14/2017", 2017L,  "Nov",     9.3,
           "MW07", "11/14/2017", 2017L,  "Nov",     9.3,
           "MW11", "11/14/2017", 2017L,  "Nov",   20.95,
           "MW14", "11/14/2017", 2017L,  "Nov",     6.2,
           "MW17", "11/14/2017", 2017L,  "Nov",     6.1,
           "MW20", "11/14/2017", 2017L,  "Nov",    3.55,
           "MW22", "11/14/2017", 2017L,  "Nov",    15.4,
           "MWAI", "11/14/2017", 2017L,  "Nov",    58.6,
           "MWBU", "11/14/2017", 2017L,  "Nov",    51.6,
           "BW02", "11/16/2017", 2017L,  "Nov",     1.1,
           "MWCI", "11/16/2017", 2017L,  "Nov",    51.6,
           "MWEU", "11/16/2017", 2017L,  "Nov",    12.5,
           "SW01", "11/16/2017", 2017L,  "Nov",       0,
           "MWDS", "11/17/2017", 2017L,  "Nov",      21,
           "MWDU", "11/17/2017", 2017L,  "Nov",      76,
         "DEPFLD",   "4/3/2018", 2018L,  "Apr",    40.7,
           "MW01",   "4/3/2018", 2018L,  "Apr",     4.5,
           "MW14",   "4/3/2018", 2018L,  "Apr",     7.4,
           "MW22",   "4/3/2018", 2018L,  "Apr",    17.2,
           "MWDS",   "4/3/2018", 2018L,  "Apr",    28.6,
           "MWDU",   "4/3/2018", 2018L,  "Apr",    79.6,
           "MWEU",   "4/3/2018", 2018L,  "Apr",    17.8,
         "DEPPBD",   "4/4/2018", 2018L,  "Apr",    23.4,
         "DEPPBS",   "4/4/2018", 2018L,  "Apr",    22.7,
           "MW06",   "4/4/2018", 2018L,  "Apr",    12.4,
           "MW07",   "4/4/2018", 2018L,  "Apr",      11,
           "MW02",   "4/5/2018", 2018L,  "Apr",    20.4,
           "MW04",   "4/5/2018", 2018L,  "Apr",    39.6,
           "MW11",   "4/5/2018", 2018L,  "Apr",    25.8,
           "MW17",   "4/5/2018", 2018L,  "Apr",     8.1,
           "MWBU",   "4/5/2018", 2018L,  "Apr",      55,
           "MWCI",   "4/5/2018", 2018L,  "Apr",    52.1,
           "BW02",   "4/6/2018", 2018L,  "Apr",     2.6,
           "MW20",   "4/6/2018", 2018L,  "Apr",     6.3,
           "MWAI",   "4/6/2018", 2018L,  "Apr",      53,
           "SW01",   "4/6/2018", 2018L,  "Apr",       0,
         "DEPFLD",  "6/18/2018", 2018L,  "Jun",    40.2,
           "MW01",  "6/18/2018", 2018L,  "Jun",     2.7,
           "MW22",  "6/18/2018", 2018L,  "Jun",   15.71,
           "MWDS",  "6/18/2018", 2018L,  "Jun",    27.4,
           "MWDU",  "6/18/2018", 2018L,  "Jun",      79,
         "DEPPBD",  "6/19/2018", 2018L,  "Jun",    22.4,
         "DEPPBS",  "6/19/2018", 2018L,  "Jun",    21.5,
           "MW02",  "6/19/2018", 2018L,  "Jun",    18.6,
           "MW06",  "6/19/2018", 2018L,  "Jun",    11.5,
           "MW07",  "6/19/2018", 2018L,  "Jun",    10.5,
           "MW17",  "6/19/2018", 2018L,  "Jun",       7,
           "MW11",  "6/20/2018", 2018L,  "Jun",    24.7,
           "MWBS",  "6/20/2018", 2018L,  "Jun",   28.01,
           "MWBU",  "6/20/2018", 2018L,  "Jun",   53.98,
           "MWEU",  "6/20/2018", 2018L,  "Jun",    17.1,
           "BW02",  "6/21/2018", 2018L,  "Jun",    1.58,
           "MW20",  "6/21/2018", 2018L,  "Jun",    6.22,
           "MWAI",  "6/21/2018", 2018L,  "Jun",   52.46,
           "MWCI",  "6/21/2018", 2018L,  "Jun",    51.9,
           "SW01",  "6/21/2018", 2018L,  "Jun",       0,
           "MW01",  "8/27/2018", 2018L,  "Aug",    0.01,
           "MW14",  "8/27/2018", 2018L,  "Aug",    5.31,
           "MW22",  "8/27/2018", 2018L,  "Aug",   13.83,
           "MWDS",  "8/27/2018", 2018L,  "Aug",   22.94,
           "MWDU",  "8/27/2018", 2018L,  "Aug",   76.18,
         "DEPPBD",  "8/28/2018", 2018L,  "Aug",    19.7,
         "DEPPBS",  "8/28/2018", 2018L,  "Aug",   19.28,
           "MW02",  "8/28/2018", 2018L,  "Aug",   14.29,
           "MW06",  "8/28/2018", 2018L,  "Aug",    9.13,
           "MW07",  "8/28/2018", 2018L,  "Aug",     8.8,
           "MW17",  "8/28/2018", 2018L,  "Aug",       6,
           "MW11",  "8/29/2018", 2018L,  "Aug",    23.1,
           "MW20",  "8/29/2018", 2018L,  "Aug",     4.3,
           "MWBS",  "8/29/2018", 2018L,  "Aug",   25.79,
           "MWBU",  "8/29/2018", 2018L,  "Aug",   51.01,
           "BW02",  "8/30/2018", 2018L,  "Aug",    0.79,
           "MWAI",  "8/30/2018", 2018L,  "Aug",   51.22,
           "SW01",  "8/30/2018", 2018L,  "Aug",       0,
           "MW01",  "12/3/2018", 2018L,  "Dec",    4.72,
           "MW14",  "12/3/2018", 2018L,  "Dec",    6.82,
           "MW22",  "12/3/2018", 2018L,  "Dec",   14.31,
         "DEPPBD",  "12/4/2018", 2018L,  "Dec",   21.78,
         "DEPPBS",  "12/4/2018", 2018L,  "Dec",   21.43,
           "MW02",  "12/4/2018", 2018L,  "Dec",   17.51,
           "MW06",  "12/4/2018", 2018L,  "Dec",   11.01,
           "MW07",  "12/4/2018", 2018L,  "Dec",   10.58,
           "MW17",  "12/4/2018", 2018L,  "Dec",     7.8,
           "MWDS",  "12/4/2018", 2018L,  "Dec",   25.13,
           "MWDU",  "12/4/2018", 2018L,  "Dec",   76.76,
           "MW11",  "12/5/2018", 2018L,  "Dec",      25,
           "MW20",  "12/5/2018", 2018L,  "Dec",    5.43,
           "MWBS",  "12/5/2018", 2018L,  "Dec",    27.3,
           "MWBU",  "12/5/2018", 2018L,  "Dec",   53.03,
           "BW02",  "12/6/2018", 2018L,  "Dec",     2.1,
           "MWAI",  "12/6/2018", 2018L,  "Dec",    52.2,
           "SW01",  "12/6/2018", 2018L,  "Dec",       0
         )
head(mass)
#> # A tibble: 6 x 5
#>   well   date        year month result
#>   <chr>  <chr>      <int> <chr>  <dbl>
#> 1 DEPFLD 11/13/2017  2017 Nov     37  
#> 2 MWBS   11/13/2017  2017 Nov     22.2
#> 3 DEPPBD 11/14/2017  2017 Nov     19.5
#> 4 DEPPBS 11/14/2017  2017 Nov     17.5
#> 5 MW01   11/14/2017  2017 Nov      4.3
#> 6 MW02   11/14/2017  2017 Nov     15.7

Created on 2021-07-29 by the reprex package (v1.0.0)

Thank you!

Your second figure isn't showing up for me, but does the plot below get close to what you're aiming for?

library(tidyverse)
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:base':
#> 
#>     date, intersect, setdiff, union

mass <- tibble::tribble(
  ~well,        ~date, ~year, ~month, ~result,
  "DEPFLD", "11/13/2017", 2017L,  "Nov",      37,
  "MWBS", "11/13/2017", 2017L,  "Nov",   22.15,
  "DEPPBD", "11/14/2017", 2017L,  "Nov",    19.5,
  "DEPPBS", "11/14/2017", 2017L,  "Nov",    17.5,
  "MW01", "11/14/2017", 2017L,  "Nov",     4.3,
  "MW02", "11/14/2017", 2017L,  "Nov",    15.7,
  "MW04", "11/14/2017", 2017L,  "Nov",      38,
  "MW06", "11/14/2017", 2017L,  "Nov",     9.3,
  "MW07", "11/14/2017", 2017L,  "Nov",     9.3,
  "MW11", "11/14/2017", 2017L,  "Nov",   20.95,
  "MW14", "11/14/2017", 2017L,  "Nov",     6.2,
  "MW17", "11/14/2017", 2017L,  "Nov",     6.1,
  "MW20", "11/14/2017", 2017L,  "Nov",    3.55,
  "MW22", "11/14/2017", 2017L,  "Nov",    15.4,
  "MWAI", "11/14/2017", 2017L,  "Nov",    58.6,
  "MWBU", "11/14/2017", 2017L,  "Nov",    51.6,
  "BW02", "11/16/2017", 2017L,  "Nov",     1.1,
  "MWCI", "11/16/2017", 2017L,  "Nov",    51.6,
  "MWEU", "11/16/2017", 2017L,  "Nov",    12.5,
  "SW01", "11/16/2017", 2017L,  "Nov",       0,
  "MWDS", "11/17/2017", 2017L,  "Nov",      21,
  "MWDU", "11/17/2017", 2017L,  "Nov",      76,
  "DEPFLD",   "4/3/2018", 2018L,  "Apr",    40.7,
  "MW01",   "4/3/2018", 2018L,  "Apr",     4.5,
  "MW14",   "4/3/2018", 2018L,  "Apr",     7.4,
  "MW22",   "4/3/2018", 2018L,  "Apr",    17.2,
  "MWDS",   "4/3/2018", 2018L,  "Apr",    28.6,
  "MWDU",   "4/3/2018", 2018L,  "Apr",    79.6,
  "MWEU",   "4/3/2018", 2018L,  "Apr",    17.8,
  "DEPPBD",   "4/4/2018", 2018L,  "Apr",    23.4,
  "DEPPBS",   "4/4/2018", 2018L,  "Apr",    22.7,
  "MW06",   "4/4/2018", 2018L,  "Apr",    12.4,
  "MW07",   "4/4/2018", 2018L,  "Apr",      11,
  "MW02",   "4/5/2018", 2018L,  "Apr",    20.4,
  "MW04",   "4/5/2018", 2018L,  "Apr",    39.6,
  "MW11",   "4/5/2018", 2018L,  "Apr",    25.8,
  "MW17",   "4/5/2018", 2018L,  "Apr",     8.1,
  "MWBU",   "4/5/2018", 2018L,  "Apr",      55,
  "MWCI",   "4/5/2018", 2018L,  "Apr",    52.1,
  "BW02",   "4/6/2018", 2018L,  "Apr",     2.6,
  "MW20",   "4/6/2018", 2018L,  "Apr",     6.3,
  "MWAI",   "4/6/2018", 2018L,  "Apr",      53,
  "SW01",   "4/6/2018", 2018L,  "Apr",       0,
  "DEPFLD",  "6/18/2018", 2018L,  "Jun",    40.2,
  "MW01",  "6/18/2018", 2018L,  "Jun",     2.7,
  "MW22",  "6/18/2018", 2018L,  "Jun",   15.71,
  "MWDS",  "6/18/2018", 2018L,  "Jun",    27.4,
  "MWDU",  "6/18/2018", 2018L,  "Jun",      79,
  "DEPPBD",  "6/19/2018", 2018L,  "Jun",    22.4,
  "DEPPBS",  "6/19/2018", 2018L,  "Jun",    21.5,
  "MW02",  "6/19/2018", 2018L,  "Jun",    18.6,
  "MW06",  "6/19/2018", 2018L,  "Jun",    11.5,
  "MW07",  "6/19/2018", 2018L,  "Jun",    10.5,
  "MW17",  "6/19/2018", 2018L,  "Jun",       7,
  "MW11",  "6/20/2018", 2018L,  "Jun",    24.7,
  "MWBS",  "6/20/2018", 2018L,  "Jun",   28.01,
  "MWBU",  "6/20/2018", 2018L,  "Jun",   53.98,
  "MWEU",  "6/20/2018", 2018L,  "Jun",    17.1,
  "BW02",  "6/21/2018", 2018L,  "Jun",    1.58,
  "MW20",  "6/21/2018", 2018L,  "Jun",    6.22,
  "MWAI",  "6/21/2018", 2018L,  "Jun",   52.46,
  "MWCI",  "6/21/2018", 2018L,  "Jun",    51.9,
  "SW01",  "6/21/2018", 2018L,  "Jun",       0,
  "MW01",  "8/27/2018", 2018L,  "Aug",    0.01,
  "MW14",  "8/27/2018", 2018L,  "Aug",    5.31,
  "MW22",  "8/27/2018", 2018L,  "Aug",   13.83,
  "MWDS",  "8/27/2018", 2018L,  "Aug",   22.94,
  "MWDU",  "8/27/2018", 2018L,  "Aug",   76.18,
  "DEPPBD",  "8/28/2018", 2018L,  "Aug",    19.7,
  "DEPPBS",  "8/28/2018", 2018L,  "Aug",   19.28,
  "MW02",  "8/28/2018", 2018L,  "Aug",   14.29,
  "MW06",  "8/28/2018", 2018L,  "Aug",    9.13,
  "MW07",  "8/28/2018", 2018L,  "Aug",     8.8,
  "MW17",  "8/28/2018", 2018L,  "Aug",       6,
  "MW11",  "8/29/2018", 2018L,  "Aug",    23.1,
  "MW20",  "8/29/2018", 2018L,  "Aug",     4.3,
  "MWBS",  "8/29/2018", 2018L,  "Aug",   25.79,
  "MWBU",  "8/29/2018", 2018L,  "Aug",   51.01,
  "BW02",  "8/30/2018", 2018L,  "Aug",    0.79,
  "MWAI",  "8/30/2018", 2018L,  "Aug",   51.22,
  "SW01",  "8/30/2018", 2018L,  "Aug",       0,
  "MW01",  "12/3/2018", 2018L,  "Dec",    4.72,
  "MW14",  "12/3/2018", 2018L,  "Dec",    6.82,
  "MW22",  "12/3/2018", 2018L,  "Dec",   14.31,
  "DEPPBD",  "12/4/2018", 2018L,  "Dec",   21.78,
  "DEPPBS",  "12/4/2018", 2018L,  "Dec",   21.43,
  "MW02",  "12/4/2018", 2018L,  "Dec",   17.51,
  "MW06",  "12/4/2018", 2018L,  "Dec",   11.01,
  "MW07",  "12/4/2018", 2018L,  "Dec",   10.58,
  "MW17",  "12/4/2018", 2018L,  "Dec",     7.8,
  "MWDS",  "12/4/2018", 2018L,  "Dec",   25.13,
  "MWDU",  "12/4/2018", 2018L,  "Dec",   76.76,
  "MW11",  "12/5/2018", 2018L,  "Dec",      25,
  "MW20",  "12/5/2018", 2018L,  "Dec",    5.43,
  "MWBS",  "12/5/2018", 2018L,  "Dec",    27.3,
  "MWBU",  "12/5/2018", 2018L,  "Dec",   53.03,
  "BW02",  "12/6/2018", 2018L,  "Dec",     2.1,
  "MWAI",  "12/6/2018", 2018L,  "Dec",    52.2,
  "SW01",  "12/6/2018", 2018L,  "Dec",       0
)


head(mass)
#> # A tibble: 6 x 5
#>   well   date        year month result
#>   <chr>  <chr>      <int> <chr>  <dbl>
#> 1 DEPFLD 11/13/2017  2017 Nov     37  
#> 2 MWBS   11/13/2017  2017 Nov     22.2
#> 3 DEPPBD 11/14/2017  2017 Nov     19.5
#> 4 DEPPBS 11/14/2017  2017 Nov     17.5
#> 5 MW01   11/14/2017  2017 Nov      4.3
#> 6 MW02   11/14/2017  2017 Nov     15.7


mass %>%
  # Create counts per month
  add_count(year, month, well) %>%
  # Fill in some zeroes
  pivot_wider(names_from = well, values_from = n, values_fill = 0) %>%
  # Return to previous format
  pivot_longer(names_to = "well", values_to = "n", cols = -c(date, year, month, result)) %>%
  # Create a "fake" date that's the first of each month to use for plotting
  mutate(temp_date = paste(year, month(mdy(date)), "01", sep = "-"),
         temp_date = ymd(temp_date)) %>%
  ggplot(aes(x = temp_date, y = well)) +
  geom_tile(aes(fill = n), colour = "white") +
  # Tell ggplot what kind of date breaks to use
  scale_x_date(date_breaks = "years", date_labels = "%Y") +
  guides(fill = guide_legend(title = "Measurements")) +
  labs(title = "Water Level Measurements Per Month",
       # I think Y is more accurate as "Well"?
       x = "Year", y = "Number of Events") +
  # This theme will be overwritten:
  # theme_bw() +
  theme_minimal() + 
  theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())

Created on 2021-07-29 by the reprex package (v2.0.0)

@cactusoxbird ---YES! This is what I am looking for.

I need to have a column for every month of the year (blank cells are important to show no data). I cannot figure out how to add this. Do I just add a dummy variable (0) in the original excel datasheet for every well and every month?

Thank you so much!!!

Ah! Yes, understood. I think that this should do it, without making you enter dummy data by hand.

mass %>%
  # Set up month as a factor. expand() below will consider
  # all levels when building out the dataframe
  mutate(month = factor(x = month, levels = month.abb)) %>%
  # Use all combinations of well/year/month
  expand(well, year, month) %>%
  # Now join this to the main dataset, with counts added by month
  left_join(x = .,
            y = mass %>%
              add_count(year, month, well, na.rm = TRUE),
            by = c("well", "year", "month")) %>%
  # Create a "fake" date that's the first of each month to use for plotting
  mutate(temp_date = paste(year, month, "01", sep = "-"),
         temp_date = ymd(temp_date),
         # Replace NAs with 0s
         n = replace_na(n, 0)) %>%
  ggplot(aes(x = temp_date, y = well)) +
  geom_tile(aes(fill = n), colour = "white") +
  # Tell ggplot what kind of date breaks to use
  scale_x_date(date_breaks = "years", date_labels = "%Y") +
  guides(fill = guide_legend(title = "Measurements")) +
  labs(title = "Water Level Measurements Per Month",
       # I think Y is more accurate as "Well"?
       x = "Year", y = "Number of Events") +
  # This theme will be overwritten:
  # theme_bw() +
  theme_minimal() + 
  theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())

Created on 2021-07-29 by the reprex package (v2.0.0)

@cactusoxbird --I got this to work! It was throwing an error, because in my file, I had month as numbers. I changed back to text, and seems to work brilliantly!!

One minor formatting question: if I want Measurements of 0 to be white, and the grid to be light grey, do I change the geom_tile(aes(fill = n), color = "white")?? It seems like the colors should be white?

Thank you so much!!!!

1 Like

Great! Yep, the color argument affects the tile outlines, so you could change it to light gray. You can also choose the start and end colors for the fill gradient manually if you'd like. For example:

mass %>%
  mutate(month = factor(x = month, levels = month.abb)) %>%
  expand(well, year, month) %>%
  left_join(x = .,
            y = mass %>%
              add_count(year, month, well, na.rm = TRUE),
            by = c("well", "year", "month")) %>%
  mutate(temp_date = paste(year, month, "01", sep = "-"),
         temp_date = ymd(temp_date),
         n = replace_na(n, 0)) %>%
  ggplot(aes(x = temp_date, y = well)) +
  # Specify the tile outlines
  geom_tile(aes(fill = n), colour = "lightgrey") +
  scale_x_date(date_breaks = "years", date_labels = "%Y") +
  guides(fill = guide_legend(title = "Measurements")) +
  labs(title = "Water Level Measurements Per Month",
       x = "Year", y = "Number of Events") +
  # Choose the ends of your gradient
  scale_fill_gradient(low = "white", high = "navyblue") +
  theme_minimal() + 
  theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())

Created on 2021-07-29 by the reprex package (v2.0.0)

1 Like

@cactusoxbird

Thanks for your help. I am throwing an error, but it may be because I modified the original data from what I reprexed here.

But, this is where the error starts:

plot6 <- mass %>%
  mutate(month = factor(x = month, levels = month.abb)) %>%
  expand(well, year, month) %>%
  left_join(x = .,
            y = mass %>%
              add_count(year, month, well, na.rm = TRUE),
            by = c("well", "year", "month")) %>%
  mutate(temp_date = paste(year, month, "01", sep = "-"),
         temp_date = ymd(temp_date),
         n = replace_na(n, 0)) %>%
  ggplot(aes(x = temp_date, y = well))
#> Error in mass %>% mutate(month = factor(x = month, levels = month.abb)) %>% : could not find function "%>%"

Created on 2021-08-03 by the reprex package (v2.0.0)

Thanks!

Initially this looks to me like tidyverse isn't loaded, so R can't access the %>% function. Are you able to get past this point if you load tidyverse or magrittr?

@cactusoxbird ---

You were correct--I have been having problems with my tidyverse/plyr disagreements!

I just ran the code, and the plot works great!

THANK YOU so much!

Craig

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.