Hi,
I am currently trying to write a shiny application within databricks which takes a few standard user inputs/filters - date, company, a upc list, etc. and then uses this to run a sql query. This query typically takes about 5minutes to run. After jamming up the cluster a few times, it came to my attention I should be using the heartbeat workaround in addition to promises and future for performance. I have implemented what I believe to be what is faithfully following the documentation, but I still receive errors. I will attempt a reprex below:
sql_string_raw <- reactive({
req(x())
req(y())
req(z())
req(a())
req(b())
req(c())
sr <- glue("FUNCTIONING_SQL_HERE"
)
sr
})
#Async Portion
raw_sql <- eventReactive(input$BeginAnalysis, {
busy(1)
val <- sql_string_raw()
future_promise({
s <- sdf_sql(sc, val)
s
}) %...>%
sdf_collect(.)
busy(0)
})
#Filter the Promise
munged_clover <- reactive({
req(raw_sql())
something <- raw_sql() %...>%
filter(!is.na(seg)) %...>%
group_by(seg) %...>%
summarize(METRICS_HERE) %...>%
ungroup(.) %...>%
mutate(METRICS_HERE) %...>%
relocate(a_column) %...>%
something
})
How is this incorrect? Any insight would be appreciated! I'm not sure if the promise is considered a logical object per the error message below.
---ERROR MESSAGE--
Use spec() to retrieve the full column specification for this data.
Specify the column types or set show_col_types = FALSE to quiet this message.
Warning: Error in as.promise.default: Don't know how to convert object of class logical into a promise
168: stop
167: as.promise.default
165: then
164: %...>%
133:
117: munged_SOMETHING
116: renderTable
115: func
102: renderFunc
101: output$SOMETHING
21: runApp
20: print.shiny.appobj
18: capture.output
Unhandled promise error: invalid connection
Unhandled promise error: invalid connection