Thanks for the tips Michael. These issues were on an AWS EC2 instance, so I was running as root and running R on the command line. I didn't have anything set with .Renviron, .Rprofile, or .R/Makevars.
I was able to resolve the problem by completely removing the /opt/R/4.2.1 directory and trying again:
# Clear out /opt/R/4.2.1
rm -rf /opt/R/4.2.1
# Install again
export R_VERSION=4.2.1
curl -O https://cdn.rstudio.com/r/ubuntu-2004/pkgs/r-${R_VERSION}_1_amd64.deb
sudo gdebi --n r-${R_VERSION}_1_amd64.deb
Then I started R with /opt/R/4.2.1/bin/R (as before) and then install.packages('dplyr') worked as expected.
I'm not sure what caused the original issue, since and I didn't make any changes to the files in the /opt/R/4.2.1 directory, but deleting all of the files and re-installing took care of it!