Error in eval: object 'variable' not found (data table feeding plot in Shiny app)

Hi there,
I have my app working, but for some reason, before displaying the plot, my app flashes up Error: object variable not found and then, a little later, draws the plot fine. The object 'variable' is a column created in the plotting data when I have melted a data table. This problem only happens when changing a particular control ('group' - which controls the dataset being plotted). I have some edited code below. I presume the plot is trying to draw before the dataset is finished being built. How do I get it to wait? I have tried req(plotdata()) within the reactive 'heatmap' section, but it doesn't seem to work.

Any suggestions would be greatly appreciated,
Will



#Load data
#=========
qlist <- fread('./App/data/qlist.csv')
grdropdown <- fread('./App/data/groupreportdropdown.csv')
reportgroups <- unique(grdropdown[,group])
qgroups <- fread('./App/data/qgroups.csv')
schools <- fread('./App/data/schools.csv')
schdists <- fread('./App/data/schoolsdistricts.csv')
ulist <- fread('./App/data/userlist.csv')
dat <- readRDS('./App/data/dat.rds')
dat <- dat[,All:='All'] #add All column for use in not analysing by a variable

#Identify user
#=============
user <- 'Joe Bloggs' #Sys.getenv("SHINYPROXY_USERNAME")
role <- ulist[email==user, role] #Sys.getenv("SHINYPROXY_USERGROUPS")

# Define UI 
#==========
ui <- fluidPage(

	  fluidRow(
	  
	    column(10, titlePanel(img(src='logo.svg', style='width:300px;', align='left'), windowTitle = 'Demo app')),
	    column(2, style = "margin-top: 50px;", actionButton('dlbutt', 'Download PDF'))
	    
	  ),
	
	  fluidRow(
	    
	    br(),
	    
	    column(3,  tagList(
	                  selectInput('group', 'Report group', choices = reportgroups),
	                  uiOutput('reportdd')
	               )
	    ),
	    
	    column(3, selectInput('by', 'Plot by', choices = c('All', 'Sex', 'Year group','Survey year'))),
	    
	    column(3, selectInput('comp', 'Comparator', choices = c('County', 'District', 'Neighbour'))),
	    
	    column(3, uiOutput('radbutt'), uiOutput('maindd'))
	           
	  ),
	  
	  plotOutput('heatmap')
	
)



# Define server logic 
#====================

server <- function(input, output) {
  
  output$reportdd <- renderUI({
    selectInput('report', 'Report', grdropdown[group==input$group, report])
  })
  
  output$radbutt <- renderUI({
    if (role=='Super'){
      radioButtons('mainchoice', 'Analysing', choices = c('District', 'School'), selected='District')
    }
  })
  
  output$maindd <- renderUI({
    if (role=='Super'){
      selectInput('main', '', choices = schdists[type==input$mainchoice][['area']])
    }
    else if (role=='GC' | role=='Partner'){
      selectInput('main', 'Analysing', choices = schdists[type=='District'][['area']])
    }
  })
  
  qvars <- reactive({
    req(input$report, input$group)
    qgroups[group==input$group & report==input$report][['bvar']]
  })
  
  pheight <- reactive({
    length(qvars())*50 + 50
  })
  
  plotdata <- reactive({
    req(input$by, input$comp)
    
    by <- qlist[qtext==input$by, var]
    compname <- input$comp
    comp <- if(compname=='County'){'*'} else {input$comp}
    compfield <- if(compname=='County'){'District'} else {input$comp}
    
    mainchoice <- if (role=='Super'){input$mainchoice} else if (role=='School') {'School'} else {'District'}
    main <- if (role=='School'){ulist[email==user, School]} else {input$main}
    
    idvars <- c('School', 'District', 'Neighbour', by) 
    
    vars <- c(idvars, qvars())
    
    a1 <- dat[, ..vars] #select the columns for this analysis
    a2 <- melt(a1, id.vars=idvars, measure.vars=qvars()) #change to long format
    a3 <- a2[get(mainchoice)==main,]
    a4 <- a3[, .N, keyby=.(by=get(by), variable, value)] #count up values by comparators
    a5 <- dcast(a4, by + variable ~ value, value.var='N', fun.aggregate = sum) #change to wide format
    a6 <- a5[, .(Y, N, Sum=sum(Y,N), Percent=Y/sum(Y, N)), keyby=.(by, variable)]#calculate percentages
    setnames(a6, old = 'by', new = paste0(qlist[var==by, qtext])) 
    a7 <- a6[, Facet:=main]
  
    c1 <- a2[grepl(comp, get(compfield)),] #create comparator dataset
    c2 <- c1[, .N, keyby=.(by=get(by), variable, value)] #count up values by comparators
    c3 <- dcast(c2, by + variable ~ value, value.var='N', fun.aggregate = sum) #change to wide format
    c4 <- c3[, .(Y, N, Sum=sum(Y,N), Percent=Y/sum(Y, N)), keyby=.(by, variable)]#calculate percentages
    setnames(c4, old = 'by', new = paste0(qlist[var==by, qtext])) 
    c5 <- c4[, Facet:=compname]
  
    plotdat <- rbindlist(list(a7, c5), use.names = T)
    plotdat$Facet <- factor(plotdat$Facet, levels=c(main, compname))
    return(na.omit(plotdat))
    
  })
  
  th <- theme_minimal() 
  
  observe({
     
    output$heatmap <- renderPlot({
      
      ggplot(plotdata(), aes(x=get(input$by), y=variable, label=paste0(round(Percent*100,1),'%'))) + 
        geom_tile(aes(fill = Percent), colour = "white", size=2) + th + 
        guides(fill='none', color='none', alpha='none') + facet_wrap(~Facet, ncol=2) +
        scale_fill_gradientn(colors=c('#f03b20', '#ffeda0', '#31a354'), values=c(0,0.5,1)) +
        geom_text(size=5) + labs(y=element_blank(), x=input$by)
      
      }
      , height=pheight() 
    )
    
  })

}


# Create Shiny object
#====================
shinyApp(ui = ui, server = server)

Just wondering, any reason you have wrapped renderPlot inside an observe?

Yes,
I previously asked on here how to get the height of the plot to depend on the number of variables in qvars(). I tried creating the pheight() reactive function, but it wasn't working when I called it from within 'heatmap'. Someone suggested that I wrap the plot in an observe, as calling pheight() was giving me an error about it not being in a reactive environment for some reason. I tried it and it worked, though I am not too clear why tbh.

Will

Ok ... maybe you need a req(pheight()) inside the observe then?

Can you provide a minimal example that illustrates the basic problem that we can actually run? I am very doubtful that putting a renderPlot() inside an observe() is a good approach — I'm reasonably sure this will end up creating multiple transient reactive output — but it's not clear that this is actually the source of the problem that you are seeing.

Yes, I'll try. Might take me a little while to get together.

Will

Hello again,
I have put this together (see below code). It doesn't give exactly the same error, but does give an error when doing the same action (changing the 'Report group' control). I think it is the same reason that the error is occurring (something to do with order of operations).
It's not very minimal, I'm afraid, but I have tried to cut some stuff out to make it shorter.

Thanks for taking the time to look at this for me. I am learning quite a lot of new things via this forum!

Will

#Load packages
#=============
pkgs <- c('shiny', 'ggplot2', 'data.table')
lapply(pkgs, library, character.only = TRUE)


#Load data
#=========
sYN <- function(x){sample(c('Y','N'), x, replace=T)}
dat <- data.table(District=rep('District A',200), School=c(rep('School A',100),rep('School B',100)),
                  Sex=rep(c('girl','boy'),100),YearGroup=rep(paste('yr',1:5),40),Q1=sYN(200),Q2=sYN(200),
                  Q3=sYN(200),Q4=sYN(200),Q5=sYN(200),Q6=sYN(200),Q7=sYN(200),Q8=sYN(200),Q9=sYN(200),
                  Q10=sYN(200),Q11=sYN(200),Q12=sYN(200),Q13=sYN(200),Q14=sYN(200),Q15=sYN(200),Q16=sYN(200),
                  Q17=sYN(200),Q18=sYN(200))
dat <- dat[,All:='All'] #add All column for use in not analysing by a 'by' variable

grdropdown <- data.table(group=c(rep('Demographics',3),rep('Health',3)), report=paste('Report',1:6))
qgroups <- data.table(group=c(rep('Demographics',9),rep('Health',9)),report=c(rep(paste('Report',1:3),3),
                      rep(paste('Report',4:6),3)), var=paste0('Q',1:18))

#Identify user
#=============
user <- 'Brian School' #Sys.getenv("SHINYPROXY_USERNAME")
role <- 'School' #Sys.getenv("SHINYPROXY_USERGROUPS")


# Define UI 
#==========
ui <- fluidPage(

	  fluidRow(
	  
	    column(10, titlePanel('Demo',windowTitle = 'Demo App')),
	    column(2, style = "margin-top: 50px;", actionButton('dlbutt', 'Download PDF'))
	    
	  ),
	
	  fluidRow(
	    
	    br(),
	    
	    column(4,  tagList(
	                  selectInput('group', 'Report group', choices = c('Demographics','Health')),
	                  uiOutput('reportdd')
	               )
	    ),
	    
	    column(4, selectInput('by', 'Plot by', choices = c('Sex','YearGroup'))),
	    
	    column(4, selectInput('comp', 'Comparator', choices = c('All', 'District')))
	           
	  ),
	  
	  plotOutput('heatmap')
	
)



# Define server logic 
#====================

server <- function(input, output) {
  
  output$reportdd <- renderUI({
    selectInput('report', 'Report', grdropdown[group==input$group, report])
  })
  
  qvars <- reactive({
    req(input$report, input$group)
    qgroups[group==input$group & report==input$report][['var']]
  })
  
  pheight <- reactive({
    length(qvars())*50
  })
  
  plotdata <- reactive({
    
    main <- 'School A'
    by <- input$by
    compfield <- input$comp
    comp <- if(compfield=='All'){'All'} else {'District A'}
    mainchoice <- 'School'
    idvars <- c('All', 'School', 'District', by)
    vars <- c(idvars, qvars())
    
    a1 <- dat[, ..vars] #select the columns for this analysis
    a2 <- melt(a1, id.vars=idvars, measure.vars=qvars()) #change to long format
    a3 <- a2[get(mainchoice)==main,]
    a4 <- a3[, .N, keyby=.(by=get(by), variable, value)] #count up values by comparators
    a5 <- dcast(a4, by + variable ~ value, value.var='N', fun.aggregate = sum) #change to wide format
    a6 <- a5[, .(Y, N, Sum=sum(Y,N), Percent=Y/sum(Y, N)), keyby=.(by, variable)]#calculate percentages
    setnames(a6, old = 'by', new = paste0(by)) 
    a7 <- a6[, Facet:=main]
  
    c1 <- a2[grepl(comp, get(compfield)),] #create comparator dataset
    c2 <- c1[, .N, keyby=.(by=get(by), variable, value)] #count up values by comparators
    c3 <- dcast(c2, by + variable ~ value, value.var='N', fun.aggregate = sum) #change to wide format
    c4 <- c3[, .(Y, N, Sum=sum(Y,N), Percent=Y/sum(Y, N)), keyby=.(by, variable)]#calculate percentages
    setnames(c4, old = 'by', new = paste0(by)) 
    c5 <- c4[, Facet:=comp]
  
    plotdat <- rbindlist(list(a7, c5), use.names = T)
    plotdat$Facet <- factor(plotdat$Facet, levels=c(main, comp))
    
    return(na.omit(plotdat))
    
  })
  
  th <- theme_minimal() 

  
  observe({
     
    output$heatmap <- renderPlot({
      
      ggplot(plotdata(), aes(x=get(input$by), y=variable, label=paste0(round(Percent*100,1),'%'))) + 
        geom_tile(aes(fill = Percent), colour = "white", size=2) + th + 
        guides(fill='none', color='none', alpha='none') + facet_wrap(~Facet, ncol=2) +
        scale_fill_gradientn(colors=c('#f03b20', '#ffeda0', '#31a354'), values=c(0,0.5,1)) +
        geom_text(size=5) + labs(y=element_blank(), x=input$by)
      
      }
      , height=pheight() 
    )
    
  })

}


# Create Shiny object
#====================
shinyApp(ui = ui, server = server)

Here is a substantially simpler app that still demonstrates the problem. Hopefully that gets you started and you can continue to simplify it still further:

library(shiny)
library(ggplot2)
library(data.table)

grdropdown <- data.table(
  group = c(rep("Demographics", 3), rep("Health", 3)), 
  report = paste("Report", 1:6)
)
qgroups <- data.table(
  group = c(rep("Demographics", 9), rep("Health", 9)), 
  report = c(rep(paste("Report", 1:3), 3), rep(paste("Report", 4:6), 3)), 
  var = paste0("Q", 1:18)
)

ui <- fluidPage(
  selectInput("group", "Report group", choices = c("Demographics", "Health")),
  uiOutput("reportdd"),
  plotOutput("heatmap")
)

server <- function(input, output) {
  output$reportdd <- renderUI({
    selectInput("report", "Report", grdropdown[group == input$group, report])
  })

  qvars <- reactive({
    req(input$report, input$group)
    qgroups[group == input$group & report == input$report][["var"]]
  })

  pheight <- reactive({
    length(qvars()) * 50
  })

  observe({
    output$heatmap <- renderPlot({
      ggplot(mtcars, aes(mpg, wt)) + geom_point()
    },
    height = pheight()
    )
  })
}

shinyApp(ui = ui, server = server)
1 Like

So, is there any way in which I can get around the problem? I only need the pheight() function because I wanted all the geom_tile rows to have the same height irrespective of the number of variables being plotted. If I can get away from having an observe around the renderPlot somehow?

Will

Would it be a good idea to use a button with an eventReactive environment, so that the dataset and plot are only built once after the button is clicked? Just wondering how I can minimise the amount of calculation and reactive output.

Will

Just to clarify, this is the actual problem, right:

library(shiny)
ui <- fluidPage(
  numericInput("height", "height", 100),
  plotOutput("plot")
)
server <- function(input, output, session) {
  output$plot <- renderPlot({
    plot(1:10)
  }, height = input$height)
}
shinyApp(ui, server)
#> Error: Operation not allowed without an active reactive context. 
#> (You tried to do something that can only be done from inside a reactive expression or observer.)

Some careful reading of the renderPlot() source code suggests that you can pass a reactive object to height:

library(shiny)
ui <- fluidPage(
  numericInput("height", "height", 500, step = 10),
  plotOutput("plot")
)
server <- function(input, output, session) {
  height <- reactive({
    input$height
  })
  
  output$plot <- renderPlot({
    plot(1:10)
  }, height = height)
}
shinyApp(ui, server)

Having figured that out, I went back and carefully read the documentation for width and height:

You can also pass in a function that returns the width/height in pixels or 'auto' ; in the body of the function you may reference reactive values and functions.

So for this case, I think the canonical answer would be:

library(shiny)
ui <- fluidPage(
  numericInput("height", "height", 500, step = 10),
  plotOutput("plot")
)
server <- function(input, output, session) {
  output$plot <- renderPlot({
    plot(1:10)
  }, height = function() input$height)
}
shinyApp(ui, server)
1 Like

I made a documentation PR so this is more clear to others in the future: https://github.com/rstudio/shiny/pull/2567

Yes, that is what was happening originally. As a result, I put the observe({...}) around the renderPlot. I think this is what is then not working well with the code that builds the dataset (using data.table). Presumably because it keeps creating transient reactive output?

In my original code I tried to use a reactive object to define the height:

pheight <- reactive({
    length(qvars())*50
  })

But without the observe({...}) around the renderPlot, it was throwing that error:

#> Error: Operation not allowed without an active reactive context.
#> (You tried to do something that can only be done from inside a reactive expression or observer.)

So, are you saying that if I called it as a function it would work without the observe({...})?...

  output$plot <- renderPlot({
    plot(1:10)
  }, height = function() pheight)

I have tried this, and although it is working for the plot height, I am still getting the following message when I change the 'Report group' control:

Warning: Error in eval: object 'variable' not found
  194: eval
  193: eval
  192: [.data.table
  190: <reactive:plotdata> [C:\Users\Will\My Tresors\Will's tresor\RM Insight\App/app.R#157]
  188: .func
  185: contextFunc
  184: env$runWith
  177: ctx$run
  176: self$.updateValue
  174: plotdata
  167: renderPlot [C:\Users\Will\My Tresors\Will's tresor\RM Insight\App/app.R#201]
  165: func
  125: drawPlot
  111: <reactive:plotObj>
   95: drawReactive
   82: origRenderFunc
   81: output$heatmap
    1: runApp

The variable column of the data table is created after melting the data table, so I'm not sure why it initially is saying it is not found and then plotting the chart a short while later.

Will

pheight is a reactive so you still need pheight() inside the function. I can't help with the other problem unless you provide a minimal reprex.

OK, I'll try to put one together as soon as I can. Bear with me, I'm not used to Shiny or creating minimal reprexs yet.

Will

This topic was automatically closed 54 days after the last reply. New replies are no longer allowed.