Welcome to rstudio community.
Could you rewrite your post (how to edit a post) so it's a reproducible example reprex? It'd be great to see the full error.
Did you have a look at the documentation for grDevices::X11? It says "This can only be done on machines/accounts that have access to an X server." Not my area of expertise, but I think macOS uses a different windowing system.
Though, it sounds like your issue has something to the documentation under type,
type- character string, one of "Xlib", "cairo", "nbcairo" or "dbcairo". Only the first will be available if the system was compiled without support for cairographics. The default is "cairo" where available except on macOS, otherwise "Xlib"."