Thanks for the advice, @stephhazlitt! I've given listing the code point ('\U03C3') a go; no dice. I'll try switching to Cairo on my Mac a go now 
EDIT: confirmed that using cairo_pdf on Mac works! Thanks a lot!
ggsave(p, filename = 'blah-raij1.pdf', device = cairo_pdf)
If I'd bothered to check the warnings when I did the test plot, I'd have seen:
There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In grid.Call(C_textBounds, as.graphicsAnnot(x$label), ... : conversion failure on '2.5σ' in 'mbcsToSbcs': dot substituted for <cf>
[etc.]
Unfortunately, it doesn't work on my Linux server, although the failure happens silently (the plot saves without a warning, but the characters are still substituted). cairo_pdf() appears to be there, but it also fails silently (plot produced but substitution occurs). ?cairo_pdf says that R will report a warning if it's not compiled with Cairo support, and that isn't happening. It also says that it defaults to Nelvetica Neue (presumably it was designed for Mac first?), so I'm wondering if it's instead a missing glyph issue on Linux.
ggsave() lets you pass arguments onto device, so I tried passing one on that I have installed locally (DejaVu Sans Mono):
ggsave(p, filename = '~/blah5.pdf', device = cairo_pdf, family = 'DejaVu Sans Mono')
Unfortunately, the font renders correctly, but the substitution is still made 
EDIT: hmm, this could be a locale problem on my server:
> Sys.getlocale()
[1] "C"