Yes my problem is that the annotation "background" layer should come first and not sure how to make ggplot work with that without the geom_blank hack but then the y/x scale would reflect x/y ranges and not just the stat layer that is added afterwards.