I can't test this since you are not providing a reprex but I think you just have to use as.Date()
annotate("text", x = as.Date("2020-05-21"), y = 0.05, label = "Current CFR Date _____: _______")
If you need more specific help, please provide a proper REPRoducible EXample (reprex) illustrating your issue.