compare output to output from previous version of package with testthat

I'm working on a package in which a function was written by an expert who has since left the project. I myself am not an expert and thus want to make sure that I never break the original algorithm without noticing.

Thus, I'd like to test the output of the current version of a function in my package against that of the same function in a previous version of the package, identifiable by a git commit, branch, or similar in the package repository.

Lets say, for example, I would like to compare the output of foo() from the dev branch against foo() from a certain commit on the main branch.

test_that("old foo matches new foo", {
  
  foo_new <- foo()
  foo_old <- ... # install and load earlier version of foo somehow
  
  expect_equal(foo_old("bar"), foo_new("bar"))
})

Can I somehow install the old version of the package temporarily to run the test and how can I avoid messing up the namespace?

Why not write tests in the old version of the package first (maybe snapshot tests, or any other unit test) and then do your edits (possibly in a git branch) and see if the tests still pass?

The question " I’m interested in your thoughts on Jenny Bryan’s question (first in the list). If you’ve got a package already, where can one start with writing tests or how should one proceed then?" towards the end of https://docs.google.com/document/d/1TLj44dNbJHeANcgV4f0JZF1-0j76EsxlxDjGBWb5rzc/ might also be useful.

Thank you! I think snapshot tests will do the trick, although it will be a bit of a hassle to retroactively add a snapshot for the function. I should have done that when I started making changes :sweat_smile:

I suppose I have to ...

  • branch from the old commit using git checkout -b add_snapshot_test <SHA of commit>
  • create a snapshot test in the branch, e.g.
testthat("foo never changes", {  
    expect_snapshot_output(foo("bar")) 
})
  • run the test once to create a snapshot
  • merge the branch with the new snapshot test (and the snapshot) into the dev branch
  • (possibly: change the foo() function call in the test to reflect changes in the function signature)

I will give it a try!

1 Like

Well, hindsight is 20/20. :grin: Good luck!