Since yesterday I try to find out why the dplyr's filter command works when explicitly naming columns but not when maping through a vector of column names (see example below).
What I am wondering about the most is the fact, that (a) the select command works in the purr context but the filter command doesn't and that (b) it throws not any error but filter returns always empty tibbles. I tired several ways unquoting the colum name using noquote() as well as bang bang (!!) but still empty tibbles. What am I missing?
Thank you very much and here is a small example with reproducible code:
library(tidyverse)
df <- tibble(a = 1:10, b = round(runif(10)), c = round(runif(10)))
## works
df %>%
dplyr::select(a, b) %>%
dplyr::filter(b == 1)
#> # A tibble: 3 x 2
#> a b
#> <int> <dbl>
#> 1 1 1.00
#> 2 7 1.00
#> 3 10 1.00
## works
map(c("b", "c"), function(x) df %>%
dplyr::select(a, x))
#> [[1]]
#> # A tibble: 10 x 2
#> a b
#> <int> <dbl>
#> 1 1 1.00
#> 2 2 0
#> 3 3 0
#> 4 4 0
#> 5 5 0
#> 6 6 0
#> 7 7 1.00
#> 8 8 0
#> 9 9 0
#> 10 10 1.00
#>
#> [[2]]
#> # A tibble: 10 x 2
#> a c
#> <int> <dbl>
#> 1 1 0
#> 2 2 1.00
#> 3 3 1.00
#> 4 4 1.00
#> 5 5 1.00
#> 6 6 0
#> 7 7 0
#> 8 8 0
#> 9 9 1.00
#> 10 10 1.00
## does not work (no error but returns empty tibbles)
map(c("b", "c"), function(x) df %>%
dplyr::select(a, x) %>%
dplyr::filter(x == 1))
#> [[1]]
#> # A tibble: 0 x 2
#> # ... with 2 variables: a <int>, b <dbl>
#>
#> [[2]]
#> # A tibble: 0 x 2
#> # ... with 2 variables: a <int>, c <dbl>
What is it that you are trying to do? The examples you show that "work" each produce different results.
You have shown us some input, df, we need to see the code you want to execute and the output you expect.
quaisiquotion is meant to be used on quosures for example:
suppressPackageStartupMessages(library(tidyverse))
df <- tibble(a = 1:10, b = round(runif(10)), c = round(runif(10)))
# v is a quosure
v <- rlang::quo(c(a, b))
# !!! converts quosure to spliced symbols,
# i.e. symbols separated by quotes
dplyr::select(df, !!!v)
#> # A tibble: 10 x 2
#> a b
#> <int> <dbl>
#> 1 1 1.
#> 2 2 0.
#> 3 3 0.
#> 4 4 0.
#> 5 5 0.
#> 6 6 0.
#> 7 7 0.
#> 8 8 1.
#> 9 9 0.
#> 10 10 1.
v <- rlang::quo(c(b, c))
dplyr::select(df, !!!v)
#> # A tibble: 10 x 2
#> b c
#> <dbl> <dbl>
#> 1 1. 1.
#> 2 0. 0.
#> 3 0. 0.
#> 4 0. 1.
#> 5 0. 1.
#> 6 0. 0.
#> 7 0. 1.
#> 8 1. 0.
#> 9 0. 1.
#> 10 1. 0.
# this fails because standard evaluation is
# used on the Collection, c(), arguments. This
# makes R look for the variables a and c in
# the current environment, but they don't exist
# there
dplyr::select(df, !!!c(a, c))
#> Error in quos(...): object 'a' not found
# typically you would use quosures in a function
# implementation
f <- function(df, columns) {
v <- rlang::enquo(columns)
dplyr::select(df, !!!v)
}
f(df, c(a, b))
#> # A tibble: 10 x 2
#> a b
#> <int> <dbl>
#> 1 1 1.
#> 2 2 0.
#> 3 3 0.
#> 4 4 0.
#> 5 5 0.
#> 6 6 0.
#> 7 7 0.
#> 8 8 1.
#> 9 9 0.
#> 10 10 1.
Thanks for your feedback. I want to have a vector for each column (b, c) containing all 1s (simply speaking, in the applied case I just keep the row IDs for those rows which works perfectly).
The code I want to execute is the unexpected using the map() command. The results of the two "works" comments differ indeed. The first one show later the result I expect (two tibbles, both only containing 1s) and the second shows that select() work the way I coded it but filter() doesn't (i.e. it's a interim result).
Here I hope (a) to get the result I expect using map(), select() and filter() and (b) to find an explanation why filter fails here.
I will dig into the use of quosure once I am back at the PC.
Sorry I don't really get what I missed with my reprex... But I try to clarify my purpose: The input is the df I create at the beginning. The function I try to write is not a function but a dplyr chain. Here's an updated reprex:
suppressPackageStartupMessages(library(tidyverse))
## create fake input data
df <- tibble(id = 1:10, a = round(runif(10)), b = round(runif(10)))
## filter column a and b seperately and
## return id where the condition is fullfilled
## works
da <- df %>%
dplyr::select(id, a) %>%
dplyr::filter(a == 1) %>%
dplyr::select(id)
db <- df %>%
dplyr::select(id, b) %>%
dplyr::filter(b == 1) %>%
dplyr::select(id)
ids <- list(da, db)
## this is what I expect as result:
## a nested list, one tibble for each column
ids
#> [[1]]
#> # A tibble: 7 x 1
#> id
#> <int>
#> 1 1
#> 2 2
#> 3 4
#> 4 5
#> 5 7
#> 6 9
#> 7 10
#>
#> [[2]]
#> # A tibble: 4 x 1
#> id
#> <int>
#> 1 1
#> 2 4
#> 3 5
#> 4 9
## the select function works without any prolems when using map
## this is just to show that filter seems to be the problematic line of code
d <- map(c("a", "b"), function(x) df %>%
dplyr::select(id, a))
## now the whole chain using map()
ids_map <- map(c("a", "b"), function(x) df %>%
dplyr::select(id, x) %>%
dplyr::filter(x == 1) %>%
dplyr::select(id))
## filter returns empty tibbles now
## this is not what I expect
ids_map
#> [[1]]
#> # A tibble: 0 x 1
#> # ... with 1 variable: id <int>
#>
#> [[2]]
#> # A tibble: 0 x 1
#> # ... with 1 variable: id <int>
The problem seems not to be a quotation issue (but I might be wrong). Your suggestion using rlang::quo is very handy but for filtering this doesn't help because the rows which were returned might be different in length and position. But getting one tibble with a column of each filter-result would be also fine instead of a nested list.
Just a note on terminology a dplyr "chain" just uses the inline function %>%.
So for example:
suppressPackageStartupMessages(library(tidyverse))
# function
f1 <- function(x, y) {
x + y
}
# a "chain" like this
1 %>% f1(2)
#> [1] 3
# is just a shorthand for calling the %>% function in the traditional way:
`%>%` (1, f1(2))
#> [1] 3
The important step you miss here is the filter() command that just keeps IDs for rows containing a 1 and drops rows containing a 0.
You are returning the full columns. My question is why the select() command picks the columns as expected while filter() doesn't filter the correct rows when using it in a purrr context - and how to solve it.
filter is working as expected. To debug map you have break it down into individual iterations to see what is going on.
# breaking map down in to iterations
map(c("b", "c"), function(x) df %>%
dplyr::select(a, x) %>%
dplyr::filter(x == 1))
suppressPackageStartupMessages(library(tidyverse))
df <- tibble(a = 1:10, b = round(runif(10)), c = round(runif(10)))
# this is the first iteration
# first step of iteration
df1 <- df %>% dplyr::select(a, "b")
# produces tibble with columns a and b
df1
#> # A tibble: 10 x 2
#> a b
#> <int> <dbl>
#> 1 1 0.
#> 2 2 1.
#> 3 3 1.
#> 4 4 1.
#> 5 5 1.
#> 6 6 1.
#> 7 7 1.
#> 8 8 1.
#> 9 9 0.
#> 10 10 0.
# second step of iteration
dplyr::filter(df1, "b" == 1)
#> # A tibble: 0 x 2
#> # ... with 2 variables: a <int>, b <dbl>
# filter doesn't find anything because "b"
# is never equal to 1
This is a great trick/workaround! Still don't get why it did work with filter when "breaking it down in to iterations" but not when using map(). The result of your approach is exactly what I want. I love gather() but never thought about to use it in this case... Thank you very much!
It's possible to get it to work that way, but since you're writing a function for a column name that's a variable, you'd have to write it in rlang syntax:
Thank you very much for the several ways to solve the problem.I will dig into them and see which one I like he most - all of them will provide the desired result!
And I definitely need to check out NSE in more detail!