Troubleshooting publishing an app, w/ private package, to Connect

colino · May 29, 2019, 2:33am

I'm publishing a Shiny app to Connect. It requires inclusion of a private package, because (a) the package is served by a private repository yet (b) Connect can't access that private repository.

The process for specifying a private package seems quite straightforward. I followed what I think are all of those straightforward steps. Still, I failed. Thanks in advance for any suggestions.

Some facts
a. The private package does not need to be compiled.
b. It imports htmlwidgets, but I've tested deployment with a separate explicit library(htmlwidgets) to no avail.
c. We've had no other problems deploying to Connect.
d. I've confirmed that other config changes actually yield changes in Connect (that is, Connect is reading the correct config file).

Here's what I did, and ended up with:

RStudio IDE 1.2 on Win10:
a. Installed all required packages including the private package (it's called RiskGauge). I installed the private package using devtools as follows:

> library(devtools)
>devtools::install_gitlab("xxxx/RiskGauge", auth_token = "xxxxxx", host="xxxxxxx")

b. tarred, then gzipped, the RiskGuage package and named it after the commit hash (Remote SHA in the DESCRIPTION file), in this case, dafc7d31fede0fd8f61b5469221f6effae23abbb.tar.gz

RStudio Connect on RHEL 7:
a. Added a setting to /etc/rstudio-connect.gcfg:

[server]
SourcePackageDir = /opt/R-packages

b. Created directory /opt/R-packages/, w/ a subdirectory /RiskGauge/ .
c. Saved my private package's .tar.gz file in /opt/R-packages/RiskGauge/
c. Set liberal permissions on /opt/R-packages/ (and subdirectories and files therein)
d. systemctl restart rstudio-connect

Back to RStudio IDE on Win10:
a. Attempted to publish the Shiny app to Connect.

Here's my log extract:

2019/05/28 22:05:37.819114530 Installing RiskGauge (1.0.0) ... 
2019/05/28 22:05:39.527271486 curl: (22) The requested URL returned error: 404 Not Found
2019/05/28 22:05:40.772211537 curl: (22) The requested URL returned error: 404 Not Found
2019/05/28 22:05:42.017255823 curl: (22) The requested URL returned error: 404 Not Found
2019/05/28 22:05:43.266277682 curl: (22) The requested URL returned error: 404 Not Found
2019/05/28 22:05:44.520517787 curl: (22) The requested URL returned error: 404 Not Found
2019/05/28 22:05:45.523664215 FAILED
**2019/05/28 22:05:45.524508485 Error in getSourceForPkgRecord(pkgRecord, srcDir(project), availablePackagesSource(repos = repos), : Failed to retrieve package sources for RiskGauge 1.0.0 from CRAN (internet connectivity issue?)**
2019/05/28 22:05:45.524551121 
2019/05/28 22:05:45.524604473 Unable to fully restore the R packages associated with this deployment.
2019/05/28 22:05:45.524609179 Please review the preceding messages to determine which package
2019/05/28 22:05:45.524638056 encountered installation difficulty and the cause of the failure.
2019/05/28 22:05:45.524641421 
2019/05/28 22:05:45.524669308 Some typical reasons for package installation failures:
2019/05/28 22:05:45.524671911   * A system library needed by the R package is not installed.
2019/05/28 22:05:45.524689386     Some of the most common package dependencies are cataloged at:
2019/05/28 22:05:45.524691039     https://github.com/rstudio/shinyapps-package-dependencies
2019/05/28 22:05:45.524707109 
2019/05/28 22:05:45.524708778   * The R package requires a newer version of R.
2019/05/28 22:05:45.524724995 
2019/05/28 22:05:45.524726563   * The C/C++ compiler is outdated. This is often true for packages
2019/05/28 22:05:45.524742661     needing C++11 features.
2019/05/28 22:05:45.524744373 
2019/05/28 22:05:45.524760346   * The R package is Windows-only or otherwise unavailable for this
2019/05/28 22:05:45.524761959     operating system.
2019/05/28 22:05:45.524777878 
2019/05/28 22:05:45.524779393   * The package is housed in a private repository that requires
2019/05/28 22:05:45.524795131     authentication to access. For more details on this, see:
2019/05/28 22:05:45.524796771     https://docs.rstudio.com/connect/admin/package-management.html#private-repositories
2019/05/28 22:05:45.524813008 
2019/05/28 22:05:45.524814668 The package description and documentation will list system requirements
2019/05/28 22:05:45.524830716 and restrictions.
2019/05/28 22:05:45.524832408 
2019/05/28 22:05:45.524848113 Please contact your RStudio Connect administrator for further help
2019/05/28 22:05:45.524849709 resolving this issue.
Build error: exit status 1
Application deployment failed with error: exit status 1
Deployment completed: https://server:443/connect/#/apps/34/logs
Warning message:
In FUN(X[[i]], ...) :
  Package 'RiskGauge 1.0.0' was installed from sources; Packrat will assume this package is available from a CRAN-like repository during future restores

cderv · May 29, 2019, 5:55am

How did you install your private package ?
From the documentation:

The package must have been installed from the git repository using the devtools package so that the hash is contained in the DESCRIPTION file on the client machine.

This is required so that the commit hash is included inside the DESCRIPTION file of the installed package. devtools (remotes) installation function does that for you. You need devtools::install_git() or other function.

If you don't do that, Rstudio connect won't know the hash to look for.

colino · May 29, 2019, 11:52am

Thanks for this - it shows an omission in my OP. I've updated it to make it clear that I installed using devtools::install_gitlab().

colino · May 29, 2019, 9:01pm

Further: I attempted publishing again (everything the same except the destination) and tailed /var/log/rstudio-connect.log as I went. The output certainly indicates failure, but it's not very verbose. Is there a more verbose log than the one I get in RStudio IDE's 'deploy' tab?

2019/05/29 16:55:07 [worker bd27bbf2; process 14551] Connecting to worker on port 33262 ...
2019/05/29 16:55:09 [worker bd27bbf2; process 14551] Connected to worker on port 33262
2019/05/29 16:56:10 [app: 35; bundle: 50] Writing bundle to: /var/lib/rstudio-connect/bundles/app-35-50.tar.gz
2019/05/29 16:56:10 [app: 35; bundle: 50] Bundle /var/lib/rstudio-connect/bundles/app-35-50.tar.gz uploaded with 8310112 bytes
2019/05/29 16:56:10 [app: 35; bundle: 50] Extracting bundle /var/lib/rstudio-connect/bundles/app-35-50.tar.gz to: /var/lib/rstudio-connect/apps/35/50
2019/05/29 16:56:11 Building Shiny application...
2019/05/29 16:56:11 Bundle requested R version 3.5.3; using /opt/R/3.5.3/bin/R which has version 3.5.3
2019/05/29 16:56:22 Build error: exit status 1

alexkgold · May 30, 2019, 4:58pm

Hi @colino,

Since you're using an RStudio Pro product, please feel free to file a formal support ticket via support@rstudio.com and we can continue to track this issue there.

When you do, please point them to this thread as you've included a lot of great information here.

Two things you could additionally provide that might be helpful is the output of
packageDescription("RiskGauge")
as well as the manifest.json generated by running rsconnect::writeManifest().

Thanks!

colino · June 1, 2019, 8:06pm

Thanks, @alexkgold! I'll probably resort to opening a ticket if I don't get this resolved w/ community help by Monday.

As you suggested, I got output of packageDescription():

> library(RiskGauge)
> packageDescription("RiskGauge")
Package: RiskGauge
Title: Risk Gauge with confidence intervals
Version: 1.0.0
Authors@R: person(given = "", family = "XXXXXXX", role = c("aut", "cre"), email = "XXXXXXX", comment = c(ORCID = "YOUR-ORCID-ID"))
Description: This package contains helper functions to create risk gauge with risk scores and confidence intervals.
License: MIT + file LICENSE
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 6.1.0.9000
Imports: htmlwidgets
RemoteType: gitlab
RemoteHost: XXXXXXX.XXXXXXX
RemoteRepo: RiskGauge
RemoteUsername: ird
RemoteRef: master
RemoteSha: dafc7d31fede0fd8f61b5469221f6effae23abbb
NeedsCompilation: no
Packaged: 2019-05-24 14:34:28 UTC; XXXXXXX
Author: XXXXXXX  [aut, cre] (<https://orcid.org/YOUR-ORCID-ID>)
Maintainer: XXXXXXX
  Built: R 3.5.3; ; 2019-05-24 14:34:29 UTC; windows```

pauleves · June 4, 2019, 11:57am

Hi, @colino Did you have any luck with this, I am facing the same issue and have followed all of the same steps. Would appreciate it if you have any new knowledge to share?

colino · June 4, 2019, 12:25pm

@pauleves, not yet. I've opened a ticket and @ian pointed out that manifest.json resulting from rsconnect::writeManifest() indicates that the package was loaded from CRAN, which is most certainly is not. So, if it's true (and it is) that the package isn't loaded from CRAN, how does Connect, or R, or the RStudio IDE come to believe that it was? I wonder whether CRAN is the residual category of some if/else block (but that's totally speculative), and becomes the value if some other conditions aren't met.

pauleves · June 4, 2019, 12:41pm

thanks @colino for the quick reply. I had also noticed that in my manifest file (and from the Write manifest warning), but had hoped that the Server.SourcePackageDir setting would overwrite that load flag.

Thanks again, please let me know if you get a good outcome.

alexkgold · June 14, 2019, 1:02pm

@colino and I spoke via RStudio support about this, so reporting back here for completeness. Packrat actually did not support GitLab packages. That support has now been added, and the updated version of Packrat (including GitLab support) will be rolled into a future release of RStudio Connect. Thanks for helping us find this issue and fix it, @colino and @pauleves!

system · July 5, 2019, 1:02pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.