I switched work computers a few days ago and installed RStudio-2021.09.2-382 and had the same problem (but for Slovenian - čšž). My new computer is running Windows build 10.0.19043.
Turning off the beta for UTF-8 on Windows only changed the behaviour in RGui - if I had the beta on, the error happened in both RGui and RStudio, if off, only in RStudio.
Setting locale Sys.setlocale("LC_CTYPE", ".1252"), did change the output of say list.files(), but it was just wrong in a different way. I tried 1252, 1250, 1251, and some others from this list Code Page Identifiers - Win32 apps | Microsoft Docs , some of them even stalled RStudio and I had to force quit. Just adding this in case it helps anyone find the issue faster.
For me the only solution that works for now is reverting to an older version (1.4.1717).