Why are tests/testthat/helper- files discouraged in testthat

Package development
#1

From https://testthat.r-lib.org/reference/test_dir.html#special-files:

Helper files start with helper and are executed before tests are run. They're also loaded by devtools::load_all() , so there's no real point to them and you should just put your helper code in R/ .

This means that any packages I use for testing only, e.g. rprojroot can no longer be suggests, but must be imports. And what about custom skip functions? They are irrelevant for the package and only used for testing.

It has been a good philosopy to separate tests from worker code. Why is there no longer a "real point" in it?

The many other changes required for testthat 3.0 are annoying and require a huge amount of work - I have no idea yet how to get rid of the many warnings created by calling third party packages.

We have decided not to use testthat in future packages, and to avoid using tidyverse in packages. tidyverse is great for shoot-and-forget reports, but it is just too much hassle to keep up with all the tibble-et-al changes.

#2

As far as I understood this section you can rename the file helper.R into setup.R?

#4

Thank for you discussion on the referenced thread. Hadley wrote

Because we're moving away from setup.R , and you really want these helpers to be available during interactive exploration with load_all() . It's not a special file — you can put it in any file in R/ .

I agree, and that's why I liked the helper-concept because these files load on load_all. But my two main questions are not adressed:

  • This violates the concept of test/testee separation
  • All helper functions must be marked import when put into /R, they are no longer together with testthat in suggests.
#8

One should never leave ones believes untested -... krrr....

I have made some robustness tests to your example https://github.com/dmenne/testpkg:

  • Using as-cran and R Cmd to be on the safe side (ok, I remember you found the opposite direction to go foul for unknown reasons)
  • Disambiguated your expect_length. I did not fully understand why you had two version of it. Was this to test a hypothesis I did not get?
  • Added an example to R code. I had expected that this definitively kill suggests for praise, but it did not.

No complaints in build!

This invalidates my suggests comment.

Nevertheless, I still do not like the concept, and do not understand what the idea was besides making package builders go crazy to follow the changes.

I still believe there should be no test code mixed with /R, even if it is not exported.

1 Like
#9

Not sure what happened with my example, so thanks for improving it! And yes there's really something to be said for experimenting. :wink:

I think you can keep using tests/testthat/helper as in the linked GitHub thread it's said such files are still supported. There's no deprecation or so.

#3

For reference https://github.com/r-lib/testthat/issues/1206

#5

Yes, but it is no longer loaded on load_all when in /testthat, so you must put in in /R to be loaded.

#7

Regarding this point, in https://testthat.r-lib.org/articles/test-fixtures.html#package I read "Setup code is typically best used to create external resources that are needed by many tests. It’s best kept to a minimum because you will have to manually run it before interactively debugging tests.
" which corroborates your statement (and makes my suggestion from earlier not a good idea at all indeed).

2 Likes
#6

  • Regarding the concept of test/testee separation I see your point but it's a different R file and it should affect the test coverage measure. Are there practical consequences of violating this concept?

  • You can keep these dependencies in Suggests. I made a very small package to convince myself. The custom expectation is defined in R/ and messages using praise. praise is listed as Suggests dependency. It passes R CMD check. https://github.com/maelle/testpkg

2 Likes