Environment:Data: data.frames expand with blue arrow but tbl_dfs don't

Alas nay:
RstudioBugStill

I know you've spent a lot of time poking at this problem already :pray: , but I think narrative descriptions of debugging experiments can be hard for others to follow (certainly my head is spinning a bit! :dizzy_face:). Can you try constructing some simple code examples that recreate the problem you're seeing and then posting that code here?

Unless the problem truly only occurs when importing data from a CSV, it's usually more helpful to use built-in data sets or ones you create in code, since this eliminates unrelated variables (but if importing does turn out to be one of the steps to recreating the problem, take a look at my code above for an example of how to do that in a "controlled experiment" kind of way).

For example, to test the hypothesis about date columns being involved, you might write some code that adds a date column to a data frame created from one of the built-in datasets — does the same issue with the missing expansion arrows occur? If so, posting that code here will be very helpful!

Will try to do so this week. In the meantime did the new preview version fix this for your reprexes? Did anyone get around to trying those files I shared?

1 Like

Thanks, @kevinushey. Sounds like my example is probably not the same problem then! :thinking: I don't want to switch RStudio versions right now because I'm finishing up a major project, but if this issue remains mysterious once that's done, I'll try the Preview release and take a stab at another reprex. This has been happening intermittently to me, too, and I've learned just how much I used the expansion triangles in my workflow :sweat_smile:.

Sorry, but I haven't yet. The truth as (as @jcblum mentioned) is that our time is quite limited and the more you can do to ease our ability to debug these problems, the better. The ideal is that you can give us a bit of standalone R code that we can copy + paste into our own RStudio sessions, and that bit of code will be sufficient to reproduce the issue.

If the code example absolutely needs to depend on external data, you can consider uploading the data somewhere publicly accessible (e.g. GitHub), and then the code provided could use download.file() to retrieve it.

I doubt it needs to depend on external data, I just don't know that i'll have much time this week to work back through scripts to see where things start to go wrong & try to recreate that with standard package data. Files are publicly shared through my google drive in the links above. It occurs to me that almost regardless of the way they were generated, a vector of Date() data >=64 length should perform the same on everybody's systems and its formatting should contain all the information required to understand the problem (i.e. there shouldn't be any 'memory' of the generation history hidden in the file). Notwithstanding the generation method may be interesting once the cause is understood, but it still shouldn't break in this manner. Cheers all.

This is my reproducible example with the blue circle issue absent.

db <- data.frame(x = 1:10, y = 2)

This particular incantation of the issue should be fixed in the preview release.

1 Like

I couldn't build a reprex since I was only getting this issue when I used datasets from a database, but I did find for some reason that adding a filter() brought the arrow back.

dataset %>%
  mutate(datetime_column = as.POSIXct(datetime_column) # blue arrow disappears

dataset %>%
  mutate(datetime_column = as.POSIXct(datetime_column) %>%
  filter(TRUE) # blue arrow comes back again

Hope this helps.

I'm using the latest preview release and am still getting the issue.

3 Likes

I can confirm that the filter(TRUE) statement brings back the blue arrow.
For now I can work with that "solution", and I hope that I don't have to explain this pipeline step to my future self.

In my current case, a dataframe produced with readxl::read_excel() has a blue arrow, whereas the same dataframe, followed by some tidyverse operations ( filter, mutate and so on) is arrowless...

I'm also having this issue and it's not fixed by downloading the preview release version. I'm not sure how to reproduce the problem, but I might provide some clues.

I have a big main dataset with a lot of factor variables and some that are numeric or character but that I treat as factors. If I import my data from csv and convert the numerical ID variable using factor() or as.factor(), the blue arrow disappears. Converting the date column using as.Date() doesn't do the same thing.

In one instance, when I use dplyr and pipes to select a few columns and filter to a few levels on one variable I still have the blue arrow. However, once I add droplevels() the arrow disappears.

I haven't used droplevels() much but I do a lot of sorting, filtering and summarising and I see the arrow disappear in a lot of my filtered and/or summarised data. I have a feeling that the issue might have something to do with how I'm treating my categorical variables but I haven't seen a definitive pattern.

Hope this helps!

Hi all,

The below produces the missing blue arrow for me and another colleague both running:
RStudio 1.2.1335
R 3.6.0
Windows 10 (64-bit)

The same issue does not arise when the code is run by another colleague running:
RStudio 1.1.463
R 3.5.2
Windows 10 (64-bit)

64 rows is the magic number. The blue arrow appears for 63 rows or less but disappears for >= 64 rows as mentioned above. The issue does not arise when using as.Date instead of as.POSIXct.

test_df <- dplyr::tibble(Date = as.POSIXct(rep("2019-02-14 00:00:00 UTC", 64)))

This is definitely the result of ALTREPs.

# 64 rows
> test_df <- dplyr::tibble(Date = as.POSIXct(rep("2019-02-14 00:00:00 UTC", 64)))
> .rs.hasAltrep(test_df)
[1] TRUE

# 63 rows
> test2_df <- dplyr::tibble(Date = as.POSIXct(rep("2019-02-14 00:00:00 UTC", 63)))
> .rs.hasAltrep(test2_df)
[1] FALSE
2 Likes

In case others are interested:

1 Like

If this is the result of ALTREPs, how do we get rid of it? Downloading the new preview version didn't work for me.

A simple reprex showing one version of the problem.

# no blue arrow
a <- iris

# blue arrow
b <- iris[1:150,]

# but... ?
identical(a, b)
#> [1] TRUE

Created on 2019-07-23 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> - Session info ----------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.0 (2019-04-26)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United Kingdom.1252 
#>  ctype    English_United Kingdom.1252 
#>  tz       Europe/London               
#>  date     2019-07-23                  
#> 
#> - Packages --------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
#>  backports     1.1.4   2019-04-10 [1] CRAN (R 3.6.0)
#>  callr         3.2.0   2019-03-15 [1] CRAN (R 3.6.0)
#>  cli           1.1.0   2019-03-19 [1] CRAN (R 3.6.0)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.6.0)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.0.2   2019-04-08 [1] CRAN (R 3.6.0)
#>  digest        0.6.19  2019-05-20 [1] CRAN (R 3.6.0)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 3.6.0)
#>  fs            1.3.1   2019-05-06 [1] CRAN (R 3.6.0)
#>  glue          1.3.1   2019-03-12 [1] CRAN (R 3.6.0)
#>  highr         0.8     2019-03-20 [1] CRAN (R 3.6.0)
#>  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.6.0)
#>  knitr         1.23    2019-05-18 [1] CRAN (R 3.6.0)
#>  magrittr    * 1.5     2014-11-22 [1] CRAN (R 3.6.0)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.6.0)
#>  pkgbuild      1.0.3   2019-03-20 [1] CRAN (R 3.6.0)
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.6.0)
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.6.0)
#>  processx      3.3.1   2019-05-08 [1] CRAN (R 3.6.0)
#>  ps            1.3.0   2018-12-21 [1] CRAN (R 3.6.0)
#>  R6            2.4.0   2019-02-14 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.1   2019-03-17 [1] CRAN (R 3.6.0)
#>  remotes       2.1.0   2019-06-24 [1] CRAN (R 3.6.0)
#>  rlang         0.4.0   2019-06-25 [1] CRAN (R 3.6.0)
#>  rmarkdown     1.13    2019-05-22 [1] CRAN (R 3.6.0)
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.6.0)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.6.0)
#>  stringi       1.4.3   2019-03-12 [1] CRAN (R 3.6.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      2.1.1   2019-04-23 [1] CRAN (R 3.6.0)
#>  usethis       1.5.0   2019-04-07 [1] CRAN (R 3.6.0)
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.6.0)
#>  xfun          0.8     2019-06-25 [1] CRAN (R 3.6.0)
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.6.0)
#> 
#> [1] C:/Users/neilc/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.0/library
1 Like

Interestingly, the symptom appears to be the same as other issues:

> .rs.hasAltrep(a)
[1] TRUE
> .rs.hasAltrep(b)
[1] FALSE

but for some reason our patch doesn't resolve this particular version of the issue.

This issue has been fixed in RStudio 1.3 for some time, but not in RStudio 1.2 (not even the preview release). We've just added the fix to RStudio 1.2 and will have a new preview up (probably sometime this week!) with the fix.

3 Likes

Update: Preview build is live. https://www.rstudio.com/products/rstudio/download/preview/

4 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.