Thank you @DavoWW. It appears, however, those suggestions on stackoverflow are for stripping UTF8 from imported data, not the R Markdown document itself.
I guess one interim solution would be to treat the R Markdown files as external text file data and run a UTF8 cleaning function on the document (code below uses this approach). Ideally, though, there would be a package / add-in so that students could easily scrub their own documents within RStudio.
# import Rmd as data
file_to_check <- readLines("path/to/my_file.Rmd")
# search for latex unfriendly characters and replace with GREMLIN
file_checked <- gsub('[^\x20-\x7E]', 'GREMLIN', file_to_check)
# identify rows with `GREMLIN`
rows_to_check <- grep("GREMLIN", file_checked)
rows_to_check
file_checked[rows_to_check]