Error when knitting RMarkdown to HTML on a networked drive

Hi,

I'm trying to knit a RMarkdown file (just the default template that you begin with as example) on a networked drive, but am getting a error which I cannot seem to solve:

image

How can I solve this error?
Info on my session:

Used RMarkdown code:


title: "testing"
author: "Koen"
date: "9-7-2020"
output: html_document

knitr::opts_chunk$set(echo = TRUE)

R Markdown

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

summary(cars)

Including Plots

You can also embed plots, for example:

plot(pressure)

Note that the echo = FALSE parameter was added to the code chunk to prevent printing of the R code that generated the plot.

Can you map the network drive to a drive letter as e.g. described here

I am having the same problem, as I believe are many other people. This issue has been open for over 9 months on stackoverflow: https://stackoverflow.com/questions/58087795/rmarkdown-not-working-on-win-10-machine-openbinaryfile-does-not-exist-no-such/62517522?noredirect=1#comment110660608_62517522

I am using the latest version of R (4.0.2), the latest version of R-Studio (1.3.1056) and pandoc 2.7.3 and it is still not working for me. My network drive is mapped to a drive letter and I have only one version of pandoc installed, the version that comes with Rstudio.

Here is the error that I am getting:

output file: test.knit.md

"C:\Users\bosr\RStudio\bin\pandoc/pandoc" +RTS -K512m -RTS test.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output test.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Users\bosr\R\R-4.0.2\library\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme:bootstrap" --include-in-header "C:\Users\bosr\AppData\Local\Temp\RtmpWiHRLj\rmarkdown-str74fc5d0457f4.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/pagebreak.lua" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/latex-div.lua" 
pandoc.exe: test.utf8.md: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversion failed with error 1
Execution halted

Here is my sessionInfo():

10:19:02  > sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] compiler_4.0.2  htmltools_0.5.0 tools_4.0.2     yaml_2.2.1      rmarkdown_2.3   knitr_1.29     
 [7] xfun_0.15       digest_0.6.25   rlang_0.4.7     evaluate_0.14   fortunes_1.5-4 
10:20:11  > rmarkdown::pandoc_version()
[1] ‘2.7.3’
10:20:38  > 

I think that the location of the pandoc executable is wrongly specified.
It seems to be "C:\Users\bosr\RStudio\bin\pandoc/pandoc" which has a mix of back and foreward slashes. And that is strange if you use the pandoc version included with RStudio.
I wonder what is your setting of the environment variable RSTUDIO_PANDOC . If that is not correct (should be like mine expect for the prefix) then you could try to set it to the correct value. My setting has only forward slashes:

Sys.getenv("RSTUDIO_PANDOC")
#> [1] "D:/tools/RStudio/bin/pandoc"

Created on 2020-07-17 by the reprex package (v0.3.0)

I understand that you did not install a pandoc version separately from the RStudio one somewhere on your system (?)

Thank you for the reply. Here is what I get:

09:20:28  > Sys.getenv("RSTUDIO_PANDOC")
[1] "C:\\Users\\bosr\\RStudio\\bin\\pandoc"

I did recently have more than one versin of pandoc installed and have since uninstalled the standalone pandoc in an effort to fix this Rmarkdown problem. I don't know what else to try.

@ HanOostdijk I defined RSTUDIO_PANDOC as an environmental variable. I am not sur why there were mixed slashes, but I have removed that environmental variable forcing Rstudio to figure out where pandoc is and now the slashes are correct, but the error is the same:

output file: test.knit.md

"C:/Users/bosr/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS test.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output test.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Users\bosr\R\R-4.0.2\library\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme:bootstrap" --include-in-header "C:\Users\bosr\AppData\Local\Temp\RtmpaW9e9v\rmarkdown-strb645ca8784f.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/pagebreak.lua" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/latex-div.lua" 
pandoc.exe: test.utf8.md: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversion failed with error 1
Execution halted

I think you eliminated now one possible source for errors.

I most often use with RStudio projects.
So I create (or reuse) a project and define an Rmd file in the project.
Questions:

  • Do you work in the same way or do you use a individual Rmd file?
    I would advise working in a project.
  • What is the result of getwd() in your project, do you see path and directory name ?
  • What happens when you use a different name than test.Rmd e.g. abc.Rmd
    (you will notice that I am becoming desperate :grinning:) ?
  • To exclude the possibility that there is something wrong in the knitr step:
    can you include the full output from the R Markdown panel (so not only the pandoc step) ?

Thanks @HanOostdijk for the followup. Most of my .Rmd files are not in projects and my wd is "M:/R_HOME" which is a network drive. I am superstitous, but maybe a littlestitus, so I tried your suggestion. I created a project named abc. Added the abc.Rmd file as the canned example you get when you create a new Rmarkdown file, and made sure my wd was set to "M:/R_HOME/abc".

13:26:50  > getwd()
[1] "M:/R_HOME/abc"

Here is the full output:

Loading required package: grDevices
Loading required package: graphics
Loading required package: stats
Loading required package: utils

Attaching package: 'zoo'

The following objects are masked from 'package:base':

    as.Date, as.Date.numeric

Loading required package: usethis

Attaching package: 'lubridate'

The following objects are masked from 'package:base':

    date, intersect, setdiff, union


Attaching package: 'testthat'

The following object is masked from 'package:devtools':

    test_file


Attaching package: 'dplyr'

The following objects are masked from 'package:plyr':

    arrange, count, desc, failwith, id, mutate, rename, summarise,
    summarize

The following object is masked from 'package:testthat':

    matches

The following objects are masked from 'package:xts':

    first, last

The following object is masked from 'package:MASS':

    select

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union


Attaching package: 'data.table'

The following objects are masked from 'package:dplyr':

    between, first, last

The following objects are masked from 'package:lubridate':

    hour, isoweek, mday, minute, month, quarter, second, wday, week,
    yday, year

The following objects are masked from 'package:xts':

    first, last

XLConnect 1.0.1 by Mirai Solutions GmbH [aut],
  Martin Studer [cre],
  The Apache Software Foundation [ctb, cph] (Apache POI),
  Graph Builder [ctb, cph] (Curvesapi Java library)
http://www.mirai-solutions.com
https://github.com/miraisolutions/xlconnect

Attaching package: 'ONJob'

The following object is masked from 'package:dplyr':

    ntile

The following objects are masked from 'package:base':

    droplevels, merge


Attaching package: 'DBI'

The following object is masked from 'package:ONJob':

    dbGetQuery

[1] "DBI Database Connections verified"

Attaching package: 'googlesheets4'

The following objects are masked from 'package:googledrive':

    request_generate, request_make

  |..........                                                            |  14%
  ordinary text without R code

  |....................                                                  |  29%
label: setup (with options) 
List of 1
 $ include: logi FALSE



processing file: abc.Rmd
  |..............................                                        |  43%
  ordinary text without R code

  |........................................                              |  57%
label: cars
  |..................................................                    |  71%
  ordinary text without R code

  |............................................................          |  86%
label: pressure (with options) 
List of 1
 $ echo: logi FALSE

  |......................................................................| 100%
  ordinary text without R code


output file: abc.knit.md

"C:/Users/bosr/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS abc.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output abc.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Users\bosr\R\R-4.0.2\library\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme:bootstrap" --include-in-header "C:\Users\bosr\AppData\Local\Temp\RtmpWcjPz7\rmarkdown-str29f87ad456b2.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/pagebreak.lua" --lua-filter "C:/Users/bosr/R/R-4.0.2/library/rmarkdown/rmd/lua/latex-div.lua" 
pandoc.exe: abc.utf8.md: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversion failed with error 1
Execution halted

Two questions/remarks:

  • the nice thing about working with a project is that there is no need to 'made sure my wd was set to "M:/R_HOME/abc" ' . Do I understand correctly that you created the project and in the opened project you added abc.Rmd ? This is more than just creating a subfolder and the file abc.Rmd !
    Maybe this in obvious to you but I want to be certain of that.
  • the output contains a lot of library statements that normally are not seen (by me at least).
    Therefore I think that these statements are caused by your R profile. See https://rstats.wtf/r-startup.html for a schema that determines where that profile (with the name .Rprofile ) is found.
    It could be that a statement in that profile somewhere overrides a setting that is used in the generation of the output file. My suggestion would be to temporarily rename the .Rprofile file.

Hope this helps. At this time no further ideas.

1 Like

Thanks for all your time. Yes, I created a project and have abc.Rproj in the abc folder. I then created the abc.Rmd file. I do have a file that is sourced by .Rprofile and loads up a list of packages I use often. I tried renaming the .Rprofile file and verified that no packages were loaded and still got the same pandoc error that abc.utf8.md was not found. I actually took a video of the file pane in Rstudio and abc.knit.md and abc.utf8.md are created at 1KB each, but immediately deleted.

Since it is not meant to be, I wrote a function that copies the Rmd file from the network drive onto the local drive, renders it, and then copies it back to the network. It probably fails in many edge cases, but it works well for my use case. I just changed all my calls to render() to render_network() and it works fine. When I run it on linux it just passes the arguments to the render() function.

# Work-around for rendering Rmarkdown files stored on network drive
render_network <- function(input, output_file = NULL, local = "c:/Users/bosr/local", overwrite = TRUE, ...) {

  if (.Platform$OS.type != "windows") {
    render(input, output_file, ...)
  } else {
    
    filename <- basename(input)
    localfile <- file.path(local, filename)
    x <- file.copy(from = input, to = localfile, overwrite = overwrite)
    # copy over all .css files as well
    ssfiles <- list.files(pattern = "\\.css$")    
    for (f in ssfiles) file.copy(from = file.path(dirname(input), f), to = dirname(localfile), overwrite = overwrite)
    if (x) {
      x <- try(render(input = localfile, output_file = output_file), ...)
      if (inherits(x, "try-error")) print("render failed")    
      x <- file.copy(from = file.path(local, output_file), to = file.path(dirname(input), output_file), overwrite = overwrite)
    }
    # Remove relevant files from local directory
    unlink(localfile)
    unlink(file.path(local, filename))
    unlink(file.path(local, output_file))
    for (f in ssfiles) unlink(file.path(local, f))
  }
  
}

Happy that this works for you.
There were also some additions to the thread https://github.com/rstudio/rmarkdown/issues/1268 lately.
But you probably saw these already.

Actually that link was very helpful. The solution from @jackhannah95 worked for me. I seems I had to set the R_USER environmental variable to "M:/" in my case and now it seems to work based on initial testing. Thanks again for your time.