I recently updated my versions of dplyr, data.table, dtplyr and have some issues with failing legacy code that mixed and matched dplyr and data.table (bad form, I know).
Is there a way to always evaluate the output of d(t)plyr operations without needing to call as.data.table/as_tibble on them?
Is there a way such that data.table objects with dplyr verbs performed on them are not implicitly converted to dtplyr objects if lazy_dt is not called?
In my use cases the dtplyr library is not called explicitly. Rather it appears that a data.table object gets implicitly converted to an unevaluated dtplyr object. Without dtplyr called into the namespace, I get the desired behaviour:
# dtplyr package not called
library(dplyr); library(data.table)
X <- data.table(pat= letters,
Bp= rnorm(26, 110, 10))
X[,z:=scale(Bp)]
Y <-X[z>.5] %>%
filter(z<3) %>%
select(pat)
# Returns normal data.table object
Implicit dtplyr conversion happens (in my actual use case I don't know how dtplyr gets called into the environment, but some dependency must call it). This behaviour is not desired.
library(data.table)
library(dplyr)
requireNamespace("dtplyr") # could also be library(dtplyr) call
x <- mtcars
setDT(x)
x[,mpg:=scale(mpg)]
x %>%
filter(mpg>1)
#>Source: local data table [5 x 11]
#>Call: `_DT2`[mpg > 1]
#>
#> mpg cyl disp hp drat wt qsec vs am gear carb
#> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#>1 2.04 4 78.7 66 4.08 2.2 19.5 1 1 4 1
#>2 1.71 4 75.7 52 4.93 1.62 18.5 1 1 4 2
#>3 2.29 4 71.1 65 4.22 1.84 19.9 1 1 4 1
#>4 1.20 4 79 66 4.08 1.94 18.9 1 1 4 1
#>5 1.71 4 95.1 113 3.77 1.51 16.9 1 1 5 2
#>
#># Use as.data.table()/as.data.frame()/as_tibble() to access results
Session Information
─ Session info ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.1.3 (2022-03-10)
os Ubuntu 20.04.4 LTS
system x86_64, linux-gnu
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Etc/UTC
date 2022-03-11
pandoc NA (via rmarkdown)
─ Packages ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date (UTC) lib source
backports 1.4.1 2021-12-13 [1] RSPM (R 4.1.0)
brio 1.1.3 2021-11-30 [1] RSPM (R 4.1.0)
cachem 1.0.6 2021-08-19 [1] RSPM (R 4.1.0)
callr 3.7.0 2021-04-20 [1] RSPM (R 4.1.0)
cli 3.2.0 2022-02-14 [1] RSPM (R 4.1.0)
crayon 1.5.0 2022-02-14 [1] RSPM (R 4.1.0)
data.table * 1.14.2 2021-09-27 [1] CRAN (R 4.1.3)
desc 1.4.1 2022-03-06 [1] RSPM (R 4.1.0)
devtools 2.4.3 2021-11-30 [1] RSPM (R 4.1.0)
digest 0.6.29 2021-12-01 [1] RSPM (R 4.1.0)
dplyr * 1.0.8.9000 2022-03-11 [1] Github (tidyverse/dplyr@8abb54b)
dtplyr 1.2.1 2022-01-19 [1] CRAN (R 4.1.3)
ellipsis 0.3.2 2021-04-29 [1] RSPM (R 4.1.0)
evaluate 0.15 2022-02-18 [1] RSPM (R 4.1.0)
fansi 1.0.2 2022-01-14 [1] RSPM (R 4.1.0)
fastmap 1.1.0 2021-01-25 [1] RSPM (R 4.1.0)
fs 1.5.2 2021-12-08 [1] RSPM (R 4.1.0)
generics 0.1.2 2022-01-31 [1] CRAN (R 4.1.3)
glue 1.6.2 2022-02-24 [1] RSPM (R 4.1.0)
highr 0.9 2021-04-16 [1] RSPM (R 4.1.0)
htmltools 0.5.2 2021-08-25 [1] RSPM (R 4.1.0)
jsonlite 1.8.0 2022-02-22 [1] RSPM (R 4.1.0)
knitr 1.37 2021-12-16 [1] RSPM (R 4.1.0)
lifecycle 1.0.1 2021-09-24 [1] RSPM (R 4.1.0)
magrittr 2.0.2 2022-01-26 [1] RSPM (R 4.1.0)
memoise 2.0.1 2021-11-26 [1] RSPM (R 4.1.0)
pillar 1.7.0 2022-02-01 [1] RSPM (R 4.1.0)
pkgbuild 1.3.1 2021-12-20 [1] RSPM (R 4.1.0)
pkgconfig 2.0.3 2019-09-22 [1] RSPM (R 4.1.0)
pkgload 1.2.4 2021-11-30 [1] RSPM (R 4.1.0)
prettyunits 1.1.1 2020-01-24 [1] RSPM (R 4.1.0)
processx 3.5.2 2021-04-30 [1] RSPM (R 4.1.0)
ps 1.6.0 2021-02-28 [1] RSPM (R 4.1.0)
purrr 0.3.4 2020-04-17 [1] RSPM (R 4.1.0)
R.cache 0.15.0 2021-04-30 [1] RSPM (R 4.1.0)
R.methodsS3 1.8.1 2020-08-26 [1] RSPM (R 4.1.0)
R.oo 1.24.0 2020-08-26 [1] RSPM (R 4.1.0)
R.utils 2.11.0 2021-09-26 [1] RSPM (R 4.1.0)
R6 2.5.1 2021-08-19 [1] RSPM (R 4.1.0)
remotes 2.4.2 2021-11-30 [1] RSPM (R 4.1.0)
reprex 2.0.1 2021-08-05 [1] RSPM (R 4.1.0)
rlang 1.0.2 2022-03-04 [1] RSPM (R 4.1.0)
rmarkdown 2.12 2022-03-02 [1] RSPM (R 4.1.0)
rprojroot 2.0.2 2020-11-15 [1] RSPM (R 4.1.0)
sessioninfo 1.2.2 2021-12-06 [1] RSPM (R 4.1.0)
styler 1.6.2 2021-09-23 [1] RSPM (R 4.1.0)
testthat 3.1.2 2022-01-20 [1] RSPM (R 4.1.0)
tibble 3.1.6 2021-11-07 [1] RSPM (R 4.1.0)
tidyselect 1.1.2 2022-02-21 [1] CRAN (R 4.1.3)
usethis 2.1.5 2021-12-09 [1] RSPM (R 4.1.0)
utf8 1.2.2 2021-07-24 [1] RSPM (R 4.1.0)
vctrs 0.3.8 2021-04-29 [1] RSPM (R 4.1.0)
withr 2.5.0 2022-03-03 [1] RSPM (R 4.1.0)
xfun 0.30 2022-03-02 [1] RSPM (R 4.1.0)
yaml 2.3.5 2022-02-21 [1] RSPM (R 4.1.0)