How to get real-time log file when running long scripts

General
1

Hi all,

I am wondering if there's any way to see/create real-time logs when running long scripts.

Many thx!

1 Like
2

There's probably a more elegant way, but this might work for you. In this case, I'm saving a new log file after each step, to avoid causing problems if you want to open up the log file while the script runs.

There might be a way to use purrr::walk or safely to have it write a single log file, but fail gracefully if you happen to have opened the log file. In that case you'd miss updates until you close it. (Unless the failed open triggered a new file name... but then maybe it just gets too messy.)

library(tidyverse)
log <- tibble(timestamp = character(), note = character())

for(t in 1:10) {
  # Do one step of your long script here
  Sys.sleep(1)

  # Add a row to the log
  log <- log %>% 
    add_row(timestamp = Sys.time() %>% as.character(), 
            note = paste("Hello step", t))

  # Save a new log file 
  write_csv(log, path = paste0("log_step",
                               stringr::str_pad(t,3,pad = 0),
                               ".csv"))
}
1 Like
3

I am using the futile.logger package to create a log file when running long scripts. With this log you can create log files or show the output in you R console. Both shows you the log entries in real time.

1 Like
4

Some logging packages:

2 Likes
5

Another package that might be interesting
https://cran.r-project.org/web/packages/loggit/index.html

3 Likes
6

Whoops! Totally thought I'd included that one.

7

Thank you very much! Very helpful to my problem.

1 Like
8

Thanks a lot! futile.logger is awesome.

9

Thank you Mara! These packages are amazing. Thanks for sharing!

1 Like
10

Thank you very much! Another amazing tool!

11

There is also the ˋloggr` :package: but still only on github

1 Like
12

The one I've actually used, of course, is also one I forgot… (>ლ)

1 Like