RStudio crashes when running c++ function in package using "Install and Restart", but works fine when .cpp file is sourced or installed from binary

Mostly just seeing if anybody understands why this is happening because I've found an (albeit slightly annoying) workaround.

I am writing my package using C++ and a set of functions (seems like those using Rcpp Armadillo, but that is just a guess) crash if I compile and install the package using the "Install and Restart" button. However, if I source the file or install it from a built binary, it works fine.

Updated to Big Sur to try and fix the problem today (made no difference).
Updated to R 4.0.3 for the same reason (still no difference)
Tried to update Xcode and Fortran installations (Xcode was already updated)

I swear this happened to me around a year ago, but I have no idea what I did then to fix it, and I would love to make sure I'm not making a stupid error.