Error: Can't combine `..1$id` <character> and `..6$id` <integer64>.

Hi everyone, I tried to load several RDS files from a certain location with this code, paste them altogether and save them as one big RDS file, with this code:

library(tidyverse)
library(dplyr)
library(readr)

rds_all <- (list.files(pattern = "*.rds", full.names = TRUE)) %>%
  map_dfr(readRDS)
saveRDS(rds_all, "All_tweets_API.rds")

I get the following error message:

Error: Can't combine `..1$id` <character> and `..6$id` <integer64>.
Run `rlang::last_error()` to see where the error occurred.

And when I run rlang::last_error(), it says the problem is

<error/vctrs_error_incompatible_type>

What happened here? And what needs to be changed, for my code to work?
My RDS files contain large amounts of Tweets.

It used to work for me before with the same code and the same amount/type of files.

I am a newbie to working with R, so any help is highly appreciated!

Welcome. It looks like in one file the id column is a character and in another, it is an integer.

You could alter the map function to convert that column so they are all the same.

I am not quite sure, if I understand your answer correctly, but should I explicitly specify the type for each column/variable? That would be a lot of work since I have 37..

Is there another way to work around it?

Is the issue just in the id column?

You can just convert that? If it is more than one, you can use functions like dplyr::across() to mutate multiple columns.

library(tidyverse)

inlist <- list(
  data.frame(a = "a", b = 1),
  data.frame(a = 1, b = 2)
)

# example of problem
inlist %>% map_dfr(~ identity(.))

(var_report <- inlist %>% map_dfr(~ summarise_all(.,
  .funs = class
)) %>% tibble())

(mismatched <- map_int(var_report, ~ length(unique(.))))
(mm_names <- names(which(mismatched > 1)))

inlist %>% map_dfr(~ mutate(.x, across(
  .cols = all_of(mm_names),
  as.character
)))

I tried to use this code snippet, but I just get the error message that

mutate no applicable method for 'mutate_' applied to an object of class "character"

Did I miss something?

The error message implies you arent passing in a list of dataframes like I do in my example.

Okay, I created a list with all the dataframes, but then I still receive the same error message 'Can't combine...'

This is the error message

> rlang::last_error()
<error/vctrs_error_incompatible_type>
Can't combine `..1$id` <character> and `..6$id` <integer64>.
Backtrace:
 1. inlist %>% map_dfr(~identity(.))
 2. purrr::map_dfr(., ~identity(.))
 3. dplyr::bind_rows(res, .id = .id)
 4. vctrs::vec_rbind(!!!dots, .names_to = .id)
 6. vctrs::vec_default_ptype2(...)
 7. vctrs::stop_incompatible_type(...)
 8. vctrs:::stop_incompatible(...)
 9. vctrs:::stop_vctrs(...)
Run `rlang::last_trace()` to see the full context.
> rlang::last_trace()
<error/vctrs_error_incompatible_type>
Can't combine `..1$id` <character> and `..6$id` <integer64>.
Backtrace:
    █
 1. ├─inlist %>% map_dfr(~identity(.))
 2. └─purrr::map_dfr(., ~identity(.))
 3.   └─dplyr::bind_rows(res, .id = .id)
 4.     └─vctrs::vec_rbind(!!!dots, .names_to = .id)
 5.       └─(function () ...
 6.         └─vctrs::vec_default_ptype2(...)
 7.           └─vctrs::stop_incompatible_type(...)
 8.             └─vctrs:::stop_incompatible(...)
 9.               └─vctrs:::stop_vctrs(...)

Yes. The part of my code where I use identity, I merely prove that I can reproduce your issue.
The code that follows that is the solution...

Sorry, if that sounds like a stupid question, but is that then the part that I need to run, after I created several dataframes?

(var_report <- inlist %>% map_dfr(~ summarise_all(.,
  .funs = class
)) %>% tibble())

(mismatched <- map_int(var_report, ~ length(unique(.))))
(mm_names <- names(which(mismatched > 1)))

inlist %>% map_dfr(~ mutate(.x, across(
  .cols = all_of(mm_names),
  as.character
)))

[/quote]

Hi @grethcool and welcome.

To build on what others have said, the dataframe columns must have the same class if you want to combine them. In other words, the id column must always be character- or integer-class.

I suspect you want id to be character class, but it does seem odd that the id might be in the form XYZ123 (character) in one file, but 123456 (integer) in another. It would be a good idea to check that there isn't a problem with the id column in some of your RDS files.

Below is a reproducible example that you could copy and paste into R on your computer to give you pointers on how to solve your particular problem.

First, we create a couple of fake dataframes and save them to a temporary location. Then we read them in with map_dfr() like you did, but we make sure this time to convert the id column to character-class so that they can all be combined.

Let me know if any of this is unclear.

# Attach packages that we'll use
library(dplyr, warn.conflicts = FALSE)
library(readr)
library(tibble)
library(purrr)

# Make two dataframes with 'id' columns of different classes
x <- tibble(id = LETTERS[1:3], value = rnorm(3))
y <- tibble(id = 1:3, value = rnorm(3))

# In dataframe 'x', the id column is character-class
x
#> # A tibble: 3 × 2
#>   id    value
#>   <chr> <dbl>
#> 1 A     0.582
#> 2 B     1.62 
#> 3 C     2.38

# In dataframe 'y', the id column is integer-class
y
#> # A tibble: 3 × 2
#>      id  value
#>   <int>  <dbl>
#> 1     1  1.07 
#> 2     2 -0.391
#> 3     3 -0.355

# You are actually combining more dataframes than this, but I've simplified
# down to just two dataframes for this example.

# You loaded your dataframes as RDS files from your computer, so let's mimic
# that here. We can save our new dataframes as RDS files to a temporary folder
# on our computer (which will be deleted when you close your R session).
temp_folder <- tempdir()
write_rds(x, file.path(temp_folder, "x.rds"))
write_rds(y, file.path(temp_folder, "y.rds"))

# Now let's get the filepaths to the two RDS files we created
rds_files <- list.files(temp_folder, pattern = "*.rds", full.names = TRUE)
  
# Below is the approach you used, which fails because the 'id' columns in the
# 'x' and 'y' dataframes are different classes (character for 'x' and integer
# for 'y'). (The '..1' and '..2' in the error messages are just the names that
# {readr} gives the dataframes if you don't supply any.)

map_dfr(rds_files, read_rds)
#> Error: Can't combine `..1$id` <character> and `..2$id` <integer>.

# So, we need to convert the 'id' column in all our dataframes to character
# class. There's a few ways to do this. One way is to use mutate() after
# read_rds() in the function that we pass to map_dfr().

# So, In the code below, map_dfr() will read each RDS file and convert the
# 'id' column to character. Once it's read all the files, it will combine them
# all. This time, the 'id' columns will both be character class and so they can
# be combined.

map_dfr(
  .x = rds_files,
  .f = ~read_rds(.x) %>% mutate(id = as.character(id))
)
#> # A tibble: 6 × 2
#>   id     value
#>   <chr>  <dbl>
#> 1 A      0.582
#> 2 B      1.62 
#> 3 C      2.38 
#> 4 1      1.07 
#> 5 2     -0.391
#> 6 3     -0.355

# Great, the 'id' columns from the dataframes in our two RDS were converted
# successfully to character class, allowing them to be combined.

# Notice that we passed to the '.f' argument of map_dfr() a function, which 
# is indicated by starting it with a '~' (tilde). We pass '.x' to read_rds()
# because the '.x' argument of map_dfr() is the vector of filepaths to our
# RDS files.

Created on 2021-10-26 by the reprex package (v2.0.1)

1 Like

Thank you so much! That works for me :slight_smile:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.