`invalidateLater` without {shiny}

[mmuurr]

I'm building a {plumber} API and I'd like to have the R process periodically update some data from an external database. Shiny handles this quite nicely with the observe({ ... invalidateLater() ... }) pattern, but I haven't found a packaged solution for Plumber (or other non-Shiny 'server' R frameworks).

I've actually been able to construct this behavior relatively easily using {later} (which both Shiny and Plumber use, via {httpuv}), but I have a few worries about which I figured I'd ask the crowd.

Here's one naïve method for implementing this type of behavior:

repeat_later <- function(f, delay) {
  force(f)
  force(delay)
  later_loop <- later::create_loop()
  later_f <- function() {
    f()
    recurse()
  }
  recurse <- function() {
    later::later(later_f, delay, later_loop)
  }
  recurse()
  later_loop  ## return later_loop so it can be destroyed
}

print_time <- function() print(Sys.time())

## this will print the time every 2 seconds.
## can terminate the background loop via the loop_handle:
##   later::destroy_loop(loop_handle)
loop_handle <- repeat_later(print_time, 2)

I'm using later::create_loop() to create a private 'loop' (in the {later} sense) specifically such that we can acquire a handle to that background loop to eventually kill it, if desired. This allows each 'repeater' to be killed separately.

I've also inspected sys.nframe(), sys.parents(), sys.calls(), and sys.frames() (from within later_f()) while this is running to try to convince myself that this method isn't building a massive call-stack tower that'll eventually hit a ceiling or tip over :-). Since the recursion is tail-recursion, that recursive overhead could be optimized away in some other environments (e.g. during C/C++ compilation), but I haven't found much documentation on whether or not this optimization exists by default in the R interpreter.

To the systems-oriented R folks here: does this seem like a valid/safe pattern? Is there something (perhaps subtle, but probably obvious in hindsight) that I've overlooked?

(Also, I tried to tag this topic with later but I can't seem to figure out how to create new tags ... is this doable?)

[meztez]

Create an update data endpoint, call the endpoint periodically using a cron job or something similar?

[nirgrahamuk]

the developer of later seems stack aware. the title of later on CRAN is
"Executes arbitrary R or C functions some time after the current time, after the R execution stack has emptied." Though I may be assuming too much. Perhaps reach out to the folks at r-lib/later: (github.com)

[mmuurr]

@nirgrahamuk I think my question is related more to the management of the recursive tail call (as an 'iterator' pattern), which isn't related specifically to {later}. But thanks, I probably will reach out to the {later} folks

[mmuurr]

@meztez but then I'd need yet another looping process somewhere else ... here I'm specifically seeking a self-contained solution; but thanks for that idea!

[nirgrahamuk]

How about simply testing your implementation aggressively.
Here's a baseline.
I can blow the stack on my laptop today with the number 3342, 1 less and it runs.
Find your number ? Then try to have later do lots of quick loops and ensure the number of loops achieved is some multiple of my simple stack exploder, and then it stands to reason that later isnt abusing your stack ?

call_self <- function(x){
  if (x>0){
    if (x %% 1000==0)
      print(x)
 call_self(x-1)
  } else {
  print("end")
  }
}

call_self(3342)

[meztez]

@mmuurr Fair enough. I've added a feature request to be able to schedule plumber endpoints using either a decorator or something similar. If you have an idea on how you would like to interface with this feature, please share.

[mmuurr]

After some further reflection (and experiments), I now see that there is no actual recursion. Each iteration simply registers the next to be run later.