Output of system call suppressed when using parallel execution

[colemonnahan]

I am developing an R package that calls an executable via a system call. One of the main advantages of using R is I can call it in parallel sessions. When I run this in the R terminal the output from the executable displays live for all parallel processes. When I try this in RStudio it neither shows live, nor at the end of execution, essentially hiding the important output from the user.

Here's a very trivial reprex:

test.fn <- function(i) {Sys.sleep(i); system('where Rterm')}
test.fn(1)

Which prints to the console

C:\Program Files\R\R-4.0.3\bin\x64\Rterm.exe

Now if I try it in parallel:

cores <- 3
library(foreach)
library(doParallel)
registerDoParallel(cores=cores)
test <- foreach(i=1:cores) %dopar% test.fn(i)

No output at all. If I run this same command in the R terminal it works fine.

> test <- foreach(i=1:cores) %dopar% test.fn(i)
C:\Program Files\R\R-4.0.3\bin\x64\Rterm.exe
C:\Program Files\R\R-4.0.3\bin\x64\Rterm.exe
C:\Program Files\R\R-4.0.3\bin\x64\Rterm.exe

I tried this with snowfall, future, and the base parallel packages and all have the same effect. No output in RStudio but works fine in the terminal. It also fails in the Rgui.

Is there any way around this without abandoning RStudio?

Thanks!

[nirgrahamuk]

try the function

test.fn <- function(i) {Sys.sleep(i); system('where Rterm', intern=TRUE)}

so that the system call returns a character vector

[colemonnahan]

I tried it and it did not work unfortunately. Neither RStudio nor Rgui. Thanks for thinking to try that.

[nirgrahamuk]

I came up with a solution that should work on windows, it involves catching any printed output with capture.output and just passing it back as data.

test.fn <- function(i) {
  what_printed <- capture.output({
    Sys.sleep(i)
    val <- system("where Rterm", intern = TRUE)
    cat("system call ", i, " shows ", val, "\n")
  })
  list(
    val = val,
    what_printed = what_printed
  )
}

cores <- 3
library(foreach)
library(doParallel)
registerDoParallel(cores=cores)
(test <- foreach(i=1:cores) %dopar% test.fn(i))

[colemonnahan]

Thanks that's a good partial solution. At least it prints the output which is valuable. I was hoping to get the updates live (e.g., it has a progress bar for each core). But this will be a reasonable compromise.