That is not the RStudio version, that is the R version. Updating RStudio will not bring any problem unless you (or any user) have custome configuration with deprecated options.
A major R version update requires recompiling of some packages so you would need to run this for each user and for your site-library if you have one
update.packages(checkBuilt = TRUE, ask = FALSE)
Also, you have to make sure the required system dependencies (for any installed package) are present on the new EC2 instance.
For this kind of task I like to use an "infrastructure as code" solution (Ansible in my case) that way I can automate any repetitive task and document the process in case anything goes wrong with the additional benefit of being reusable if I need to do this again in the future.