How can I filter an arrow table based on whether a list column contains certain values?
For example, I'd like to filter the
dd arrow table to keep all rows where
y contains either 2 or 4.
library(tidyverse) library(arrow) # data dd <- tibble( x = 1:3, y = c(list(1:3), list(2:5), list(c(1L, 5L))) ) %>% arrow_table()
I'd like this filtering to occur before
collect() is run. The desired output from above example is the first two rows (since y contains 2 or 4 for those two rows).