You might need to re-mark the encoding on some character vectors, e.g. (assuming they truly are UTF-8)
Encoding(cv) <- "UTF-8"
However, there are a number of assorted issues re: the handling of UTF-8 encoding (and printing of UTF-8 characters) on Windows, so it's possible that everything is indeed encoded and 'working' correctly; it's simply not printing correctly. Unfortunately, many of these issues are R issues rather than RStudio issues and so any associated fixes would need to be implemented upstream.