Is it possible to use variables set in index.Rmd in the other Yaml files?

Ideally I'd like to be able to set up a new book just by modifying one file: index.Rmd, as opposed to having to scan through and set things in _output.yml and _bookdown.yml, too.

However, though r is evaluated in index.Rmd, even in the yaml part, I can't seem to reference those values in _output.yml, for example.

(An alternative might be if perhaps the Yaml values themselves are available to use elsewhere. Is this possible? For example, what if I want to use the author or title or description somewhere else in the book?)

I am sure to understand well what you want to do. Can you give a dummy example of the type of value you want to set in _bookdown.yml and _output.yml ?

In a Rmd file, the value of the yaml header should be available in rmarkdown::metadata object from within the document. Try to run this

---
title: my title
author: someone
output: html_document
---

```{r}
str(rmarkdown::metadata)
```

and you'll see the result

1 Like

I can't seem to get it working in either file. I can reference these values in other Rmd files, but I don't think I can in the yml files.

One thing I'd like to do is programmatically set the book file name in _bookdown.yml.

I've tried, but so far been unsuccessful to even use R code in _bookdown.yml.

For example, I tried using this in my _bookdown.yml file:


book_filename: "`r rmarkdown::metadata$title`"

I also tried setting a parameter in index.Rmd and referencing it like so:


book_filename: "`r params$bookFileName`"

I'm thinking maybe _bookdown.yml is used to create the render command itself, meaning nothing is available to it yet.

Either way, can you think of a way I can set the book filename programmatically?

I don't think _output.yml is used prior to build, so it's a bit more confusing why I can't get it to work there. I'm trying to set the "edit" link in the bookdown::gitbook: config section. First I tried this without success:


edit: "https://github.com/orgname/`r params$project`/edit/master/%s"

Next I tried the following:


edit: "`r cat('https://github.com/orgname/', params$project, '/edit/master/%s')`"

But I get the following error when I try to click the edit link in the Rstudio browser:


`/rmd_output/1/%60r%20cat('https://github.com/orgname/',%20params$project,%20'/edit/master/index.Rmd')%60 not found`

It appears to me right now that I can't use R code at all in either _bookdown.yml or _output.yml?

I just tried updating all my packages and R and R Studio, and I still can't seem to get anything out of index.Rmd and into _bookdown.yml or _output.yml.

Apparently R just isn't able to run in either of those files? Any other ideas of how I could do this?

This is indeed tricky, because a lot of things are mixed up in bookdown.

A bit of context first:

This kind of syntax will only work if the file is knit.

"`r rmarkdown::metadata$title`"

This is an inline r expression you can use in an rmarkdown file. And it works in the yaml header because the file is knitted before pandoc process it.

Inside another yaml file, it won't work because this external yaml is not process by knitr.

However, you can run expression in yaml using this syntax !expr. This is a feature from the yaml :package: that rmarkdown & friends is using with eval.expr set to TRUE

yaml::yaml.load("test: !expr 1+1", eval.expr = TRUE)
#> $test
#> [1] 2

Created on 2020-09-09 by the reprex package (v0.3.0.9001)

But for that to work, the r object need to be available.

We could try to use

book_filename: !expr rmarkdown::metadata$title

but rmarkdown::metadata needs to be filed up for that to work and I am not sure it will work.

This is an interesting use case. For now I believe you'll need to use workaround. And I could think of several but only the first one is working

  1. You can modify the yaml config file before rendering your book to modify the content programmatically. You can even use a temp one or have several yaml you would select when rendering

  2. You could try to set a R variable in the session to be used when loading the yaml file that contains a !expr call. You need to use clean_envir = FALSE for now (but it wil be deprecated)

  3. We could do some adjustment in bookdown. One example is can pass a list of bookdown configuration in render_book - This work a that way for output_format. See below.
    We need to open an option to discuss that.

Example for solution 1

bookname <- "mybook"
ori_yml <- "_bookdown.yml"
tmp_yml <- tempfile(fileext = ".yml")
yml_content <- yaml::read_yaml(ori_yml)
yml_content[["book_filename"]] <- bookname
yaml::write_yaml(yml_content, tmp_yml)
bookdown::render_book(
  "index.Rmd",
  output_format = "bookdown::epub_book",
  config_file = tmp_yml
)

Example for solution 2

Write this in your _bookdown.yml yaml

book_filename: !expr bookname

and use this code to render

bookname <- "mybook"
bookdown::render_book(
  "index.Rmd",
  output_format = "bookdown::epub_book",
  clean_envir = FALSE
)

I set output_format in both example just to show the the ebook name is changed as expected.

About edit configuration, you could apply one of this strategy too because you can set it in the _bookdown.yml too (see https://bookdown.org/yihui/bookdown/configuration.html)

About the modification of _output.yml option

You could use output_options argument to change that IMO. It works well for rmarkdown, and it works also for all output format. That means you can change that at rendering using somethink like this

bookdown::render_book(
  "index.Rmd",
  output_format = "bookdown::gitbook",
  output_options = list(
    config = 
      list(edit = "https://github.com/orgname/myproject/edit/master/%s"))
)

This would modify the config value if gitbook() format.

With this answer, I have just wanted to share some workaround. It is definitly not obvious at first.

I would like to have your opinion on this. I think it could be improved in some way for _bookdown.yml option to work as the modification of output options. Something like

bookdown::render_book(
  "index.Rmd",
  output_format = "bookdown::gitbook",
  config_options = list(book_filename = "my_book")
)

That would be merge with the config file used.

I eager to hear your feedback on this. And thanks a lot for all the feedback you gave us on bookdown !

1 Like

Fantastic stuff here. I'll mess around with it all and give feedback afterwards. Thanks again.

1 Like

Ok, I'm confused. I would have thought this meant I could use it in the yaml in index.rmd, but I've got exactly the same trouble.

I moved the toc and edit config settings into index.rmd, and they work fine, but I can't process R code in there still. Using the yaml expression syntax you mentioned above doesn't work, either.

It seems to work on other yaml values in index.rmd, but not those ones. Is there something special happening with those that prevents them from being processed still, even though they are in an rmd file?

:thinking: This would mean those values are retrieved from the YAML file before any knitting happens. And that is possible in the case of bookdown where a lot can go on before any rendering.
Just to be clear: The syntax for inline code is a knitr feature. So the file must be knitted so that the value are replaced, then the yaml header should be read. This happens in Rmarkdown - that is why it is working.

Does !expr syntax from yaml :package: works ? Do you have a working example with the demo book or another to confirm this ?

I understand that this is rather confusing to not have one way of doing thinks. Maybe !expr is safest to use in yaml because it will be process always when reading the yaml ?

I've tried this a few times, and I still feel like I must not understand how all of the processing steps come together. Here's a link to one example of what I've tried. You can see the code I'm trying on line 23 of index.rmd. I try to reuse "bookname" as the edit link, but it seems to return NULL when I'm in the "output" section. If I try to use it earlier in the YAML (before the "output" section but after "bookname"), such as the date, it just seems to pass the command through to become the value in the HTML.

Or perhaps I'm still just messing up the syntax?

No I think what you are trying to do is very depend on when the code is evaluated and by which tools.

rmarkdown::metadata will be filled only during the rendering of the Rmd file by rmarkdown. Before rendering or after rendering, bookdown will do some stuff to get information for gitbook configuration, but those value in the yaml are not read during rendering. That is why rmarkdown::metadata$title is NULL.

Also, you wrote edit: !expr rmarkdown::metadata$title but edit must be a link :thinking: So I am really not sure on what is your expected output.

I think the best way currently to achieve your worflow is by making a _main.R or _render.R that will do some pre processing step to fill your files (as with templates), or pass arguments to functions. (as shown here above)

It is a worflow pretty specific and I am not sure it would be easy to offer a generic solution to that.

I tried forming a link, first, but since it wasn't working, I began to second-guess my string-building syntax. I assumed these were all just strings, so I just wanted to see if I could get anything to show up in there, so I decided to just use the very simplest command you had given above.

Regardless, I guess the basic answer remains the same, that the variables aren't available yet?

Yeah, it looks like it. I've been trying to avoid doing this really for one reason only. I wanted the "build book" button in Rstudio to continue working. I can give that up, but before I do I guess it's worth asking if there's any way to override its behavior so it still works if I do the above?

If you call your script _render.R and it is at the root of the project it will use this file to render when clicking on Build book button. This may be an undocumented behavior but you can find examples and see it in the source code:

The button behavior is controled by the site: field you have in your index.Rmd yaml header, and by default it is bookdown::bookdown_site

So you can use a Makefile or a _render.R script to make everything you want for the rendering, and still get the button to work.

You could also create your own *_site function, but that is more complex.

Hope it helps. Those *down tools have a lot of hidden gems like that in their source code :sweat_smile: , I guess on purpose for advanced use to find them !

1 Like

Frankly, as soon as I saw the name '_render.R' I had a sneaking suspicion something like this was the case. lol. Thanks for all the help. Going to get started on it.

So, if I call 'bookdown::renderbook(...)' in a continuous integration script, will it also use my new _render.R file, or do I need to change that command somehow?

No. You'll need to call your script directly in the CI worflow using Rscript --quiet _render.R for example. You can also use a Makefile. And that is better to do that in a CI that open R and execute command.

_render.R is not called when using render_book - it is in a special render function inside bookdown_site.

1 Like

Thanks! I'll play around with it all some more.