As @ron shows, a scale_* function can work. But the coord_* functions do something similar. The differences are:
-
scale_* will remove any observations that don't fall within the limits. This could affect other calculations.
-
coord_* will just restrict the limits of what's drawn. Nothing from the dataset is ignored for calculations.
You can actually see this with your example:
ggplot(Df, aes(b, a)) +
geom_boxplot() +
coord_cartesian(ylim = c(-2.5, 5)) +
labs(title = "Using coord_*")
ggplot(Df, aes(b, a)) +
geom_boxplot() +
scale_y_continuous(limits = c(-2.5, 5)) +
labs(title = "Using scale_*")

Edited to help comparison
The top of the boxes are somewhat different. It's not much, but it shows the point.