Binary packages for different R versions in RStudio Package Manager?

The R versioned Docker images (https://github.com/rocker-org/rocker-versioned2) use the RStudio Package Manager snapshots as their baseline. The default configuration installs binary packages. However, it seems that using any R version other than the one running on the RStudio Package Manager results in issues like this:

docker pull rocker/r-ver:4.0.2
docker run -it --rm rocker/r-ver:4.0.2
## From inside R
install.packages("coda")
library(coda)
#> Warning message:
#> package ‘coda’ was built under R version 4.0.3

Installing the packages from the source snapshot works without problems:

install.packages("coda", repos = "https://packagemanager.rstudio.com/all/344")
library(coda)
# No warnings

Is the official guidance that any R version other than the latest (R 4.0.4 at the time of this writing) should use source package installs instead of binary? Or are there plans to support binary packages for multiple R versions?

See related issue here: https://github.com/rocker-org/rocker-versioned2/issues/127