Are other git actions in that directory similarly slow? What happens if you try to run
git status
from the terminal in that project directory?
You can also try running:
Sys.setenv(RSTUDIO_GIT_LOG = "1")
to see what git actions RStudio is taking behind the scenes, to see which of those in particular are slow (if that is indeed the case).