Understanding R Markdown Output Code

R Studio Community

Recently with some updates I can no longer knit R Markdown/R Notebooks as I am receiving a pandoc.exe, openBinaryFile: does not exist error. I suspect it has something to do with where the various files are located on my computer (i.e. pandoc files on the logical drive, R library files external network folder). My suspicion is based upon the fact that I initially could not save files from RStudio but then in the Options dialog changed the default working dir from "~" to "H:/My Documents" rather than using the actual path. Is there somewhere where I can read to understand the R Markdown output. For example (this is from a working knit)

output file: test.knit.md

"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS test.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output test.html --lua-filter "C:\Users\reichmaj\Documents\R\win-library\4.0\rmarkdown\rmarkdown\lua\pagebreak.lua" --lua-filter "C:\Users\reichmaj\Documents\R\win-library\4.0\rmarkdown\rmarkdown\lua\latex-div.lua" --self-contained --variable bs3=TRUE --standalone --section-divs --template "C:\Users\reichmaj\Documents\R\win-library\4.0\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable theme=bootstrap --include-in-header "C:\Users\reichmaj\AppData\Local\Temp\RtmpMbpOmq\rmarkdown-strb60237456fa.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"

Output created: test.html

So that I can understand what the output is telling me.

Jeff

1 Like

We have several issues open regarding Pandoc and Network drive. Among those this one: pandoc: openFile: does not exist (No such file or directory) · Issue #1268 · rstudio/rmarkdown · GitHub

From the command you linked above, which file is possibly on a network drive ?

I see mainly path starting with C:\

Is it test.Rmd that is on network drive ?

R version 3.6.3
RStudio version 1.2.5042
Pandoc.exe is located C:/Program Files/RStudio/bin/pandoc
R Libraries (win-library) located networked drive

The r Markdown code I posted is from a working knit. I'm unable to post the knit code that's failing (it on a secure network). That is why I was asking if there is some sort of output description detailing the code output.

It also maybe an older version of pandoc. I'm looking into seeing it we can't get a newer version.

1 Like

This could be the common issue if I reread the comments on the issue thread I linked too.

Is this configuration new ? Was it working before ?
Is the network drive mounted on a Drive with a letter on your windows computer ?

Yes it was working before (Win 7), started since they (IT) upgraded to Windows 10 version 1909. Yes the network drive is mounted (assigned) a letter (H:/ My Documents/R/win-library/3.6 drive ) in my case. Other wise its a long long string. I suspected this was a problem for RStudio when I was receiving a "The filename, directory name, or volume label syntax is incorrect" error and read on this blog where some one suggested using the actual drive letter. After changing the default working directory from ("~") to to H: drive that issue seems to have been resolved.

So do you mean that this is Rstudio IDE issue with default working dir setting ?
That is interesting !

Also do you have the command line for this error ? Even if you need to sanitize it a bit - it would help understand where Pandoc command line went to look.

Is your default user document folder on Windows set on H: drive ? I wonder if that could cause issue with Pandoc

Christophe,

I've been having a similar problem, but hadn't gotten around to posting a question about it. After reading this thread, I also tried changing the default directory and I was able to knit rmarkdown documents successfully.

In my current set-up, I remote into a secure cloud server, which is where I do all my R coding. But all my files (personal packages, R projects, etc.) are stored on a separate network drive. The default directory path for a project looks like this:

"\\\\securefiles/mydepartment/Users/Joel/projects/myproject"

When I try knitting a document to PDF, I get the following error:

Error: pandoc document conversion failed with error 1

But the securefiles network drive is already mapped to drive letter Z: on the cloud server. So I changed the default directory to this:

"Z:/mydepartment/Users/Joel/projects/myproject"

After this change, I was able to knit the document without error.

My question is, how do I get RStudio to always use the mapped Z: drive as the default directory path?

Not sure how to answer that. Bottom line is I have two issues that my IT folks can't seem to resolve.

(1) The RStudio IDE would not allow me to save *.R or *.Rmd files. I read were the UNC may not like the actually file path to external network locations and if there is a letter associated with the path to use that drive letter as the default working directory. Whereas by default the working directory in the Tools - Options is ~ I reset mine to use the drive letter (and path) and that seems to have taken care of the saving issue.
(2) The other issue is a (apparently) Pandoc.exe in that RStudtio can't find the location. Not sure best but best guess. I'll check into the link you originally sent tomorrow when I get into work. I also would if it how my IT folk update our systems (remotely) had if it might not be better if they physically load R/RStudio as an administer to insure links are made. But my guess is it has to do with the funky path names and maybe R Studio just gets confused. I did notice that the knit output doesn't use the disk letter code path but rather the full path names. So again I'm wondering if its a path names issue.

I'll see if I can't at lest re-type the knit output tomorrow

1 Like

Here is the problematic R Notebook code:

output file: lkj-.knit.md

"C:/Program Files/RStudio/bin/pandoc" +RTS -K512m lkj-.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output lkj-,html --email-obfuscation none --self-contained --standalone --section-divs --template "\coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme"bootstrap" --include-in-header "C:\Users\al52975\AppData\Local\Temp\RtmpmQKdWo\rmarkdown-strdb07d0bf7d.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --lua-filter "\coe.ic.gov/Home/ESDC/nga_home_052/al52975/My Documents/R/win-library/3.6/rmarkdown/rmd/lua/pagebreak.lua" --lua-filter "\coe.ic.gov/Home/ESDC/nga_home_052/al52975/My Documents/R/win-library/3.6/rmarkdown/rmd/lua/latex-div.lua"
pandoc.exe: \: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversaion failed with error 1
Execution halted

There is some thought that the following path "\coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html" used by RStudio is incorrect it should be "\coe.ic.gov\Home\ESDC\nga_home_052\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html". Is there a way to change or programmatically change that to see

Christophe

How do you change the default directory? Do you change it via the Tools-Global Options-Default working directory ......

@reichmaj thank you that is interesting. I feel the issue is with this network path pass as argument to Pandoc

--template "\coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html"

basically you rmarkdown :package: installation is on another network drive and Pandoc does not find this path to be correct. Is this path above correct and working when you paste it in an explorer window ?

I don't know if it is Pandoc that does not deal correctly or rmarkdown that does not create the correct command line to pandoc. Or even RStudio IDE which does something different with this path thing.

I know Pandoc has fixed some issues in the past, and I think you are using an old version because still using RStudio 1.2.
can you check ?

rmarkdown::pandoc_version()

You can use a recent version of Pandoc by following

That would be the first thing to do to rule Pandoc version.

Other check would be to see what these 2 commands returns

system.file("rmarkdown", "lua", "latex-div.lua", package = "rmarkdown")
rmarkdown::pkg_file_lua("latex-div.lua")

can you run them ?

Also on your computer, does H:// is for \coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\ ?

Then if you're willing to help more, we could try several things like just running Pandoc command line to see how it works or try to enter debug mode to see how the path is maybe modified.

My IT folks seem to think the problem maybe associated with

--template "\coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html"

they say the actual path should be

--template "\coe.ic.gov\Home\ESDC\nga_home_052\al52975\My Documents\R\win-library\3.6\rmarkdown\rmd\h\default.html"

However both paths take me to the same html page.

rmarkdown::pandoc_version()

[1] '2.7.2'

system.file("rmarkdown", "lua", "latex-div.lua", package = "rmarkdown")

[1] ""

assuming I typed correctly

rmarkdown::pkg_file_lua("latex-div.lua")

Error : 'pkg_file_lua' is not an exported object from 'namespace:rmarkdown'

yes H: == \coe.ic.gov\Home\ESDC\NGA_HO~2\al52975\ actually is should be
`\coe.ic.gov\Home\ESDC\nga_home_052\al52975" my IT folks seem to think the "NGA_HO~2" is the issue?????

Pandoc version could be involved or not - Last version 2.11.4. You can download one to use or upgrade RStudio 1.4 which has a newer version. I don't know if it is the problem but just saying if you want to try

And I forgot to asked you the rmarkdown version you were using... it seems not a recent one. I should have guessed it with the path above - sorry
So we'll simplify can you just run to see what path R use for the package ?

system.file(package = "rmarkdown")

Paths with ~ are usually short path name on Windows (see utils::shortPathName() in R).
We'll try something to see if this is related. Not sure it will work though.
I'll answer back while you check the above

Here is a code to try and see what works and does not work - hope you'll understand it to modify it if I missed something about your environment.
I have put this to run in a temp directory but you also not run the 2 first commands and 2 last to execute this in the same work dir as your R Markdown file.

dir.create(tmp_dir <- tempfile())
owd <- setwd(tmp_dir)

xfun::write_utf8(c(
  "---", 
  "title: test",
  "---", 
  "",
  "# dummy",
  ""
), "test.md")

convert <- function(template) {
  rmarkdown::pandoc_convert(
    "test.md",
    to = "html",
    from = "markdown",
    output = "test.html",
    verbose = TRUE,
    options = c("--template", template)
  )
}
# with path computed from R not modified by rmarkdown - should fail ?
convert(system.file("rmd/h/default.html", package = "rmarkdown"))
# with path computed from R but modified by rmarkdown - should work ?
convert(rmarkdown::pandoc_path_arg(system.file("rmd/h/default.html", package = "rmarkdown")))
# manual path - short path with ~
convert("\\coe.ic.gov\\Home\\ESDC\\NGA_HO~2\\al52975\\My Documents\\R\\win-library\\3.6\\rmarkdown\\rmd\\h\\default.html")
convert("//coe.ic.gov/Home/ESDC/NGA_HO~2/al52975/My Documents/R/win-library/3.6/rmarkdown/rmd/h/default.html")
# manual path - long path without ~
convert("\\coe.ic.gov\\Home\\ESDC\\nga_home_052\\al52975\\My Documents\\R\\win-library\\3.6\\rmarkdown\\rmd\\h\\default.html")
convert("//coe.ic.gov/Home/ESDC/nga_home_052/al52975/My Documents/R/win-library/3.6/rmarkdown/rmd/h/default.html")
# manual path - using letter
convert("H:\\My Documents\\R\\win-library\\3.6\\rmarkdown\\rmd\\h\\default.html")
convert("H:/My Documents/R/win-library/3.6/rmarkdown/rmd/h/default.html")

# reseting work dir an cleaning
setwd(owd)
unlink(tmp_dir, recursive = TRUE)

ok I'll give your code a try. What about the direction of back slashes // verse \

ok I'll give your code a try. What about the direction of back slashes / verse \

Never mind it would appear that is what you are asking me to do in the code

Windows can work with both I think and I know there could be issues so I think it worth a try with both. To write the paths you are mentioning above in Understanding R Markdown Output Code - #13 by reichmaj with \...\...\ , this need to be escaped in R using \\ that is why there is two instead of one. Because of R escape for backslash in R.

Don't forget to check this please:

system.file(package = "rmarkdown")

so that we know how R return you network path exactly

Is this code run from within said R Notebook or external fro the R script command line?

You should run it from an R script in your R console. Not inside a Rmd file.