How to Efficiently Separate Base Plot Function and Data/Layer Defintion (ggplot2)

Concept
Often times I produce many plots with the same base layout (time series, scatter, etc.). The data structure and thus column names do however differ. Also plots may have different auxiliary layers such as trend lines, labels, etc. Therefore I thought I might be a good idea to generate a generic plot function (scatter in this case) and feed it with additional layers as appropriate.

In the example below I use a plotting function plot_scatter and a library lib.layer. Important to note: they are defined before I even know how the data set looks like. Within the function call I define how the plot is meant to look like. I pick two additional layers from the library (layer.mean, layer.label) as well.

Question
So far so good. I fear that present example does not fully benefit from the various possibilities of non-standard-evaluation in R (and ggplot2). Especially the substitute calls in lib.layer and subsequent eval_tidy in the function don't feel right. Maybe someone resolved a similar problem differently or has suggestions how to improve current approach.

Cheers, Matt

library(tidyverse)
data(mtcars)

# Layer Library
lib.layer <- list(
  layer.mean = substitute(geom_hline(aes(yintercept = !!sym(.value)),
    data = .data %>% 
      group_by(!!sym(.group)) %>% 
      summarise(!!sym(.value) := mean(!!sym(.value))))),
  layer.label = substitute(geom_text(aes(label = !!sym(.group)), colour = "black"))
)

# Plot Function (Scatter Plot)
plot_scatter <- function(.data, x, .value, .group, .lay){
  e <- environment()
  ggplot(.data, aes(!!sym(x), !!sym(.value), colour = factor(!!sym(.group)))) +
    geom_point(size = 6) + # Base layer
    map(.lay, rlang::eval_tidy, env = e) # Evaluate layers (lib.layer)
}

# Pick layer from the library
lay.pick <- lib.layer[c("layer.label", "layer.mean")]

# Plotting
p <- plot_scatter(mtcars, x = "disp", .value = "hp", .group = "cyl", .lay = lay.pick)
print(p)

Created on 2018-11-04 by the reprex package (v0.2.1)