If your df$Date column is already of Date class, use as.numeric(df$Date), otherwise convert the column to Date class first, then use as.numeric. In your example, "3/24/2020" is a character string, so let's work with that in the example below.
x = "3/24/2020"
x
#> [1] "3/24/2020"
class(x)
#> [1] "character"
as.numeric(x)
#> Warning: NAs introduced by coercion
#> [1] NA
x = lubridate::mdy(x)
x
#> [1] "2020-03-24"
class(x)
#> [1] "Date"
x = as.numeric(x)
x
#> [1] 18345
y = as.character(x)
y
#> [1] "2020-03-24"
class(y)
#> [1] "character"
But do you need to convert your dates to numeric at all? ggplot handles date columns without the need to convert them to numeric values.
As suggested by the code above, Dates in R are actually just numeric values with a Date class attached. For example:
y=100
y
#> [1] 100
class(y) = "Date"
y
#> [1] "1970-04-11"
as.numeric(as.character(df$Date)) is necessary for converting factors back to their underlying categorical values. If df$Date is of Date (or character) class, you'll see that if you run as.numeric(as.character(df$Date)) you'll get a vector of missing values. The reason for this is as.character converts the dates to character strings and as.numeric returns missing values when applied to character strings. The first code block shows this.