Rstudio Connect - Scheduled Report Failure

Hi,

I have an instance of Rstudio Connect, which has been working very well for our organization. Recently though, our scheduled reports have stopped being update daily. I have refreshed the reports in question and reset the the scheduling parameters as they should be. This effort has not resolved the issues. Has anyone come across this before? Any advice would be greatly appreciated!

Cheers,
Tim

Very strange! Have the reports been executing? I.e. have you noticed any side effects (emails, etc.)? Are you "publishing" the report after output (i.e. can you see the historical executions and the data is only staying the same)?

Basically, I see some matrix of the following options:

  • Report is scheduled to run daily
  • Report is executing daily but output is not being published to Connect
  • Report is executing daily and output is being published to Connect
  • Report is executing daily but data is not changing when it should be

The only other option would be that the report is scheduled but is not executing at all (suggesting a Connect bug). I have never seen such a case, but it would be important to fix if so!

Is it possible to share some more detail about (1) what you mean by "not being updated daily", (2) the scheduling, and (3) your RStudio Connect version number?

An example scheduling pane. The "Publish output after it is generated" part is very important if you want the updated output on your server. You might check the "History" tab and the "Logs" pane to see if you can track down the "bad" renders and ensure that they are executing.

Hi Cole,

The reports are scheduled to run daily at 0400 hrs and to be published after it is generated. This has been working smoothly for the last year. Recently, the reports have stopped executing daily and being published to Connect. I've rescheduled the reports and refreshed the reports. The refreshed reports reflect the current data holdings. When I revisited the report days later, the report hasn't been updated. When I look at the report history, the only record present is the version generated when I manually refreshed the report.

We are using RStudio Connect v1.6.6.1-4. Looking at the audit logs, it shows that the "Obsolete rending remover" user has decreased in frequency and is only present when reports are manually refreshed.

I have a meeting scheduled with the Server Admin tomorrow, so hopefully I have some additional information to share.

Thanks for the quick response to my posting!
Tim

Very troubling! Do you see any entries in the Logs pane that suggest the report tried to render or anything? Would you mind sharing a screen shot of the "scheduling" pane much like I did above?

If this is something you can't track down yourself or needs a bit of sensitivity, feel free to open a support ticket by emailing support@rstudio.com - as a professional customer, you are entitled to pro support! However, if there is any progress we can make here, then that is definitely valuable as a public resource as well :smile:

The admin should see lines like this in the RStudio Connect log file (/var/log/rstudio-connect.log) during an execution that may provide additional input, as well.

2019/01/22 01:47:43 [document: 1983; variant: 2115; schedule: 173] Scheduling next run for 2019-01-23 01:46:52 +0000 UTC
2019/01/22 01:47:46 [document: 1983; variant: 2115; schedule: 173] Beginning scheduled document execution
2019/01/22 01:47:46 [document: 1983; variant: 2115; schedule: 173] Running document
2019/01/22 01:48:03 [document: 1983; variant: 2115; bundle: 24643] Completed scheduled document execution
2019/01/22 01:48:03 [document: 1983; variant: 2115; schedule: 173] Completed scheduled document execution
2019/01/22 01:48:18 [document: 1081; variant: 1175; schedule: 122] Scheduling next run for 2019-01-22 02:48:14.412 +0000 UTC

One of my coworkers also suggested having your admin check the email configuration for your server, sendmail, SMTP, and the like. If the report is failing, it should be sending emails about the failures. Are you receiving emails with output / errors for other reports, or is this the only problem report?

Hi Cole,

I just proxied into the email account and noticed the following error for one of the reports:

There was an error loading the selected job
{"data":"","status":500,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"url":"/api/applications/193/job/zrqznWytFn8Gqkm1","headers":{"Accept":"application/json, text/plain, /","X-RSC-XSRF":"pQIWxMG7hjhTJcXDxSZDAnyNgamCidhEXXdE4n47KZI="}},"statusText":""}

I forgot to include the requested screen shot earlier as requested above, so here it is:

Hi Cole,

sorry for the slow response! MY IT support thought of this as well, and sent this along:

I had BIS check the SMTP server to see if they saw any e-mails coming from the RStudio server. This is what they saw:

08:13:35 78E0 DMN: MSG 46183074 Accepted connection: [10.177.10.33] (ihispharm1.xxxx.ca) 08:13:35 78E0 DMN: MSG 46183074 Receiving file: E:\GTWY3\wpgate\GWIA\receive\f2f754c5.920 08:13:36 78E0 DMN: MSG 46183074 SMTP session ended: [10.177.10.33] (ihispharm1.island_health.ca) 08:13:39 6DB4 MSG 46183074 Processing inbound message: E:\GTWY3\wpgate\GWIA\receive\f2f754c5.920 08:13:39 6DB4 MSG 46183074 Sender: rstudioHIU@xxxx.org 08:13:39 6DB4 MSG 46183074 Recipient:redacted_name@xxxx.org 08:13:39 6DB4 MSG 46183074 Queuing to MTA 08:13:39 6DB4 MSG 46183074 File: E:\GTWY3\wpgate\gwia\wpcsin\4\4c457f33.co1 Message Id: (76916930.23A:175:33338) Size: 2.4 Mb

Then I checked the Rstudio log for that time period and this was in the log...

2019/01/21 08:13:25 [document: 105; variant: 91; schedule: 28] Scheduling next run for 2019-01-22 12:12:35.957 +0000 UTC
2019/01/21 08:13:30 [document: 217; variant: 122; bundle: 765] Message recipients: redacted_name@xxxx.org
2019/01/21 08:13:30 /connect/src/connect/mailsender/mailsender.go:183: (mail-sender) Sending mail from ["Redacted Name" <rstudioHIU@xxxx.org>] to
2019/01/21 08:13:30 /connect/src/connect/mailsender/mailsender.go:167: (mail-sender) SMTP: Sending mail with configuration (wiped password): {Host:gwia-apps.xxxx Port:25 SSL:false StartTLS: InsecureSkipVerify:false Username: Password:}
2019/01/21 08:13:30 /connect/src/connect/mailsender/sendmail.go:68: (mail-sender) SMTP: Connecting to gwia-apps.peigov:25
2019/01/21 08:13:35 /connect/src/connect/mailsender/sendmail.go:76: (mail-sender) SMTP: Establishing client against gwia-apps.peigov:25
2019/01/21 08:13:35 /connect/src/connect/mailsender/sendmail.go:125: (mail-sender) SMTP: Issuing command MAIL rstudioHIU@xxxx.org
2019/01/21 08:13:35 /connect/src/connect/mailsender/sendmail.go:137: (mail-sender) SMTP: Issuing command RCPT redacted_name@xxxx.org
2019/01/21 08:13:35 /connect/src/connect/mailsender/sendmail.go:144: (mail-sender) SMTP: Sending data ...
2019/01/21 08:13:36 /connect/src/connect/mailsender/sendmail.go:150: (mail-sender) SMTP: Issuing command QUIT
2019/01/21 08:13:36 /connect/src/connect/mailsender/sendmail.go:154: (mail-sender) SMTP: Closed client against gwia-apps.peigov:25
2019/01/21 08:13:36 /connect/src/connect/mailsender/sendmail.go:154: (mail-sender) SMTP: Disconnected from gwia-apps.peigov:25
2019/01/21 08:13:36 /connect/src/connect/mailsender/mailsender.go:194: (mail-sender) Sent mail from ["Redacted_Name" <rstudioHIU@xxxx.org>] to

So, it looks like this part is working! We're still investigating this issue.

Cheers,
Tim

Very interesting! Thanks for the update, and glad to hear Connect is trying to send the emails it should be! Immediately before the blurb in the logs that you sent, you should have seen "Beginning scheduled document", "Running document" and "Completed scheduled document." Was that the case?

It is concerning to me that the output is not published onto Connect, though! I was hoping that this might be a simple issue, but I think it might be best if you forward a link to this discussion on to our support team at support@rstudio.com so we can handle this more formally! As a pro customer, our support staff is best equipped and happy to help with these types of issues!

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.