I my case knitting a Rmd file with a graph always produces an intermediary folder with the graph.
The folder name is the name of the Rmd file followed by _files . Note the underscore.
When I knit (use knit btton in RStudio) atest.Rmd with the following contents
---
graphics: yes
output:
pdf_document:
keep_tex: true
keep_md: true
---
```{r sinus-plot, echo=FALSE,fig.cap="a caption"}
plot(sin(seq(-pi, pi, length.out = 100)))
```
this will result in the files atest.md, atest.tex and the file atest.pdf with the graph and without a visible error.
The folder ./atest_files/figure-latex (. is the folder with atest.Rmd . ) contains the file sinus-plot-1.pdf .
The relevant contents of atest.md :

The relevant contents of atest.tex :
\begin{figure}
\centering
\includegraphics{atest_files/figure-latex/sinus-plot-1.pdf}
\caption{a caption}
\end{figure}
So in my case I have (see) no problems with an underscore.
My sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.0.0 htmltools_0.4.0 tools_4.0.0 yaml_2.2.1 Rcpp_1.0.4.6 rmarkdown_2.1 knitr_1.28 xfun_0.13
[9] digest_0.6.25 rlang_0.4.5 evaluate_0.14