expect_error - expected and actual errors look identical but test fails

[selkamand]

Hi,

I've got a function that throws an error (expected). Despite the expected and actual messages looking identical, the test fails due to the 'unexpected' message.

I think it might have to do with the deparse(substitute(x)) call since when I avoid getting the expression as a string, the test works as expected

library(testthat)
library(cli)

assertdataframe <- function(x) {
  
  string_argname <- deparse(substitute(x))
  
  if (!is.data.frame(x)) {
    cli_abort("Error: '{string_argname}' is not a data frame! Object class: {class(x)}")
  }
  invisible(TRUE)
}

expect_error(assertdataframe(list(x = 1:5, y = 6:10)), "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list")
#> Error: `assertdataframe(list(x = 1:5, y = 6:10))` threw an error with unexpected message.
#> Expected match: "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list"
#> Actual message: "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list"

^{Created on 2023-01-03 with reprex v2.0.2}

Any ideas about why this test fails?

[nirgrahamuk]

cli uses unicode characters in the error text to perform formatting (on my rstudio setup, a yellowish exclamation at the beginning).
This is exlipicit for me when I run your code (my 'Actual message') is different from yours, but is what I would expect and would explain your issue; its strange to me that it doesnt show for you.

> expect_error(assertdataframe(list(x = 1:5, y = 6:10)),
+              "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list")
Error: `assertdataframe(list(x = 1:5, y = 6:10))` threw an error with unexpected message.
Expected match: "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list"
Actual message: "\033[38;5;255mError: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list\033[39m"

A cursory look shows that cli package includes test_that_cli() and that this uses snapshots

[selkamand]

Thanks @nirgrahamuk !

That helps a lot. Not 100% sure for the reason for our difference. I'm on linux, so maybe that contributes.

At first I assumed it was the unicode characters causing problems too, but I just realised that even if the unicode chars are included in the actual message, my test should still pass because it 1:1 matches a subset of the actual error message (expect_error uses pattern matching)

Turns out the real problem is parenthesis are treated as special regex characters by expect_error, so i just need to add fixed=TRUE to get the proper behaviour.

library(testthat)
library(cli)

assertdataframe <- function(x) {

  string_argname <- deparse(substitute(x))

  if (!is.data.frame(x)) {
    cli_abort("Error: '{string_argname}' is not a data frame! Object class: {class(x)}")
  }
  invisible(TRUE)
}

expect_error(assertdataframe(list(x = 1:5, y = 6:10)), "Error: 'list(x = 1:5, y = 6:10)' is not a data frame! Object class: list", fixed=TRUE)

^{Created on 2023-01-04 with reprex v2.0.2}

Thanks for your time (and happy new year )