This is related to R, not RStudio, if you are on windows, each R version gets its own package library, that is why the new version can't see your old packages.
After a major R update, you have to reinstall all packages anyway because binaries are compiled for a different R version, so even an update (with checkBuilt = TRUE) will actually reinstall all packages.
You can change the default library folder by setting your R_LIBS_SITE environmental variable on a .Reviron or .Rprofile file, for example:
# In a .Renviron file you can set it by adding a line like this one but with your desired library path
R_LIBS_SITE="C:\\Program Files\\R\\R-4.1.0\\library"
For a more detailed explanation, you can read this blog post