Sorting data changes its number of rows

[meitei]

Hi All,
I have a time series as attached
in the link https://drive.google.com/file/d/1mPTUGBpd8rvmAurHZZWKeLWEaPrFSEN9/view?usp=sharing

I used following code to sort it which led to reduce in its number of row/length in this case.
Could anyone explain it or suggest how to sort it so as to have the same number of rows.

Thanks

time<- read_csv('time.csv', col_names =TRUE)
t <- as.POSIXct(time$datetime, format="%Y-%m-%d %H:%M:%S")
length(t)
t1 <- sort(c(t))
length(t1)

[AlexisW]

It's due to the conversion:

table(is.na(t))
# FALSE   TRUE 
#183956     21

time[is.na(t),]
# A tibble: 21 x 1
#   datetime          
#   <chr>             
# 1 2030-03-1002:00:00
# 2 2031-03-0902:00:00
# 3 2032-03-1402:00:00
# 4 2033-03-1302:00:00
# 5 2034-03-1202:00:00
# 6 2035-03-1102:00:00
# 7 2036-03-0902:00:00
# 8 2037-03-0802:00:00
# 9 2038-03-1402:00:00
#10 2039-03-1302:00:00
# ... with 11 more rows

Not sure why, I suspect this has to do with timezone switching:

as.POSIXct("2039-03-13 02:00:00", format="%Y-%m-%d %H:%M:%S")
# [1] NA
as.POSIXct("2039-03-13 01:00:00", format="%Y-%m-%d %H:%M:%S")
# [1] "2039-03-13 01:00:00 EST"
as.POSIXct("2039-03-13 03:00:00", format="%Y-%m-%d %H:%M:%S")
# [1] "2039-03-13 03:00:00 EDT"

The package lubridate tends to be more robust to these things:

library(lubridate)
time <- time %>%
  mutate(converted_POSIXct = as.POSIXct(datetime, format="%Y-%m-%d %H:%M:%S"),
         converted_dt = as_datatime(datetime)) %>%
  arrange(converted_dt)

table(is.na(time$converted_POSIXct))
# FALSE   TRUE 
#183956     21 
table(is.na(time$converted_dt))
# FALSE 
#183977

[siddharthprabhu]

I'm unable to see a difference in the number of rows.

library(readr)

time <- read_csv("time.csv", col_names = TRUE)
#> Parsed with column specification:
#> cols(
#>   datetime = col_character()
#> )

t <- as.POSIXct(time$datetime, format = "%Y-%m-%d %H:%M:%S")

length(t)
#> [1] 183977

t1 <- sort(c(t))

length(t1)
#> [1] 183977

^{Created on 2020-08-19 by the reprex package (v0.3.0)}

[meitei]

I am not sure what led to this.BUt, still i have the same error.