How are you installing R? Are you installing from source, with multiple versions side-by-side?
RStudio Server Pro supports multiple versions, and we usually recommend adding versions of R, not replacing R. https://docs.rstudio.com/ide/server-pro/r-versions.html#installing-multiple-versions-of-r
Re: user library paths, you shouldn't need to do anything special - R will automatically create a different library for each version on R on Linux (this behavior is different on desktops). For example, without any settings, I have:
# in a session using R 3.4.4
.libPaths()
# "/home/sean/R/x86_64-pc-linux-gnu-library/3.4" "/opt/R/3.4.4/lib/R/library"
# in a session using R 3.5.0
.libPaths()
# "/home/sean/R/x86_64-pc-linux-gnu-library/3.5" "/opt/R/3.5.0/lib/R/library"
This is without setting r-libs-user at all.
Finally, you might find this information useful if you are also managing a site library: