How to stop 'lubridate' applying local time

[trador]

Hi,

I have lots of data collected in UTC. I would like to be able to apply some sunrise calculations on them but I am having an issue with 'lubridate' converting my time stamps into BST despite setting them as UTC (I am in the UK so I think this is being picked up from my computers system). How can I get lubridate to ignore my system time and stick with UTC. I have tried with_tz but this doesn't seem to have helped.

Advice on what is going wrong would be greatly appreciated.

Here is an example of my data format and the steps I am taking to create a time stamp:

library(data.table)
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:data.table':
#> 
#>     hour, isoweek, mday, minute, month, quarter, second, wday, week,
#>     yday, year
#> The following objects are masked from 'package:base':
#> 
#>     date, intersect, setdiff, union

dat <- data.table(Date_M = as.integer(c(20180115, 20180615, 20190115, 20190615, 20200115)),
                     Time_M =c( "00:00:49", "00:00:49", "00:00:49", "00:00:49", "00:00:49"))

str(dat$Date_M) # int [1:28809] 20180113 
#>  int [1:5] 20180115 20180615 20190115 20190615 20200115
str(dat$Time_M) # chr [1:28809] "03:00:49.0270"
#>  chr [1:5] "00:00:49" "00:00:49" "00:00:49" "00:00:49" "00:00:49"

# Convert Date_M to POSIX
dat <- dat[, Date_M := ymd(Date_M, tz="UTC")] # convert to POSIXct using lubridate
dat <- dat[, datetime := paste0(Date_M," ", Time_M)] # Make a combined Character col of date and time
dat <- dat[, datetime := with_tz(ymd_hms(datetime), tz="UTC")] # convert to POSIXct using lubridate maintaining tz as UTC

Looking at the table it appears fine:

knitr::kable(dat)

Date_M	Time_M	datetime
2018-01-15	00:00:49	2018-01-15 00:00:49
2018-06-15	00:00:49	2018-06-15 00:00:49
2019-01-15	00:00:49	2019-01-15 00:00:49
2019-06-15	00:00:49	2019-06-15 00:00:49
2020-01-15	00:00:49	2020-01-15 00:00:49

But if i try and select one of the 'dat' rows using the datetime, it becomes clear that for dates in summer are having BST applied to them.

# call the row for "2018-01-15 00:00:49"...all good
dat[datetime=="2018-01-15 00:00:49"] 
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

# call the row for "2018-06-15 00:00:49"...empty
dat[datetime=="2018-06-15 00:00:49"]                                          
#> Empty data.table (0 rows and 3 cols): Date_M,Time_M,datetime

# call the row for "2018-06-15 00:00:49" but advance it by 1 hour...
dat[datetime=="2018-06-15 01:00:49"] 
#>        Date_M   Time_M            datetime
#> 1: 2018-06-15 00:00:49 2018-06-15 00:00:49

^{Created on 2022-11-08 by the reprex package (v2.0.1)}

How can i force lubridate (or any other POSIX time stamps) to stay in UTC and ignore my system time?

For info I am using data.table version 1.14.0, lubridate version 1.7.10.

Thank you

[FJCC]

I thing what you need to do is force the time used in the comparison to be in UTC.

library(data.table)
library(lubridate)

dat <- data.table(Date_M = as.integer(c(20180115, 20180615, 20190115, 20190615, 20200115)),
                  Time_M =c( "00:00:49", "00:00:49", "00:00:49", "00:00:49", "00:00:49"))

# Convert Date_M to POSIX
dat <- dat[, Date_M := ymd(Date_M, tz="UTC")] # convert to POSIXct using lubridate
dat <- dat[, datetime := paste0(Date_M," ", Time_M)] # Make a combined Character col of date and time
dat <- dat[, datetime := with_tz(ymd_hms(datetime), tz="UTC")] # convert to POSIXct

dat[datetime=="2018-01-15 00:00:49"] 
#> Empty data.table (0 rows and 3 cols): Date_M,Time_M,datetime

dat[datetime=="2018-01-14 17:00:49"]  #Adjust for my tz
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

dat[datetime== ymd_hms("2018-01-15 00:00:49", tz = "UTC")] 
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

^{Created on 2022-11-08 with reprex v2.0.2}

[trador]

Hi and thank you for your response.

In theory I could try to correct for my time zone, but my issue is that I did not collect the data in the UK. The point of collecting it in UTC was to avoid these this summertime/timezone issues. I am also performing a lot of calculations based on time so would like to keep it consistently in UTC.

I am wondering if there is a way to stop lubridate from applying my local time in the first place?

[FJCC]

I don't thing lubridate is applying your local time. Your data.table is in UTC. What is applying local time is R's conversion of strings like "2018-01-15 00:00:49". When you test

datetime=="2018-01-15 00:00:49"

datetime is in UTC but the string gets converted to BST by as.POSIXct() or some similar function. You can set one of R's environment variables to use UTC as the default timezone , as shown in the code below. If you change the default in the code, you will have to do it for every session. You should be able to store that setting in the .Renviron file and make it automatic for every session.

library(data.table)
library(lubridate)

dat <- data.table(Date_M = as.integer(c(20180115, 20180615, 20190115, 20190615, 20200115)),
                  Time_M =c( "00:00:49", "00:00:49", "00:00:49", "00:00:49", "00:00:49"))

# Convert Date_M to POSIX
dat <- dat[, Date_M := ymd(Date_M, tz="UTC")] # convert to POSIXct using lubridate
dat <- dat[, datetime := paste0(Date_M," ", Time_M)] # Make a combined Character col of date and time
dat <- dat[, datetime := with_tz(ymd_hms(datetime), tz="UTC")] # convert to POSIXct

#as.POSIXct() uses my local  time: MST
as.POSIXct("2022-01-01 12:23:21") 
#> [1] "2022-01-01 12:23:21 MST"

#2018-01-15 00:00:49 in MST: datetime is UTC 
dat[datetime=="2018-01-15 00:00:49"] 
#> Empty data.table (0 rows and 3 cols): Date_M,Time_M,datetime

#2018-01-14 17:00:49 in MST: datetime is UTC 
dat[datetime=="2018-01-14 17:00:49"] 
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

#lubridate's ymd_hms() function defaults to UTC
dat[datetime== ymd_hms("2018-01-15 00:00:49")] 
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

#Set R's default time zone to UTC
Sys.setenv(TZ='GMT')

#now as.POSIXct() defaults to UTC
as.POSIXct("2022-01-01 12:23:21") 
#> [1] "2022-01-01 12:23:21 GMT"

#2018-01-15 00:00:49 and datetime in UTC
dat[datetime=="2018-01-15 00:00:49"] 
#>        Date_M   Time_M            datetime
#> 1: 2018-01-15 00:00:49 2018-01-15 00:00:49

^{Created on 2022-11-09 with reprex v2.0.2}

[trador]

Thank you, this was exactly what I was looking for. It has also cleared up some of my confusion regarding where the 'local' time was being applied in the functions.

@FJCC

#Set R's default time zone to UTC
Sys.setenv(TZ='GMT')