DESCRIPTION file different between CRAN and RSPM causing installation failures for certain packages

[davidkim83]

I've noticed certain packages failing to install from RSPM, while it succeeds for CRAN.

• DistributionUtils
• GeneralizedHyperbolic
• SkewHyperbolic

It seems that, when the original DESCRIPTION file's Encoding field is latin1 (or maybe something not UTF-8), installation fails.

Maybe RSPM (rightfully) changes it to UTF-8 in constructing its own repository, but it's not actually converting the original source files from the stated encoding to the new one.

Can anyone suggest a solid workaround for this?

Test Code:

pkg <- "DistributionUtils"

# [FAILS]
# Error: input string 1 is invalid UTF-8
install.packages(pkg, repos="https://packagemanager.posit.co/cran/latest")

# [SUCCEEDS]
# but cannot use RStudio's Help pane
install.packages(pkg, repos="https://packagemanager.posit.co/cran/latest", INSTALL_opts="--no-docs")
# Returns Encoding="UTF-8"
desc::desc_get("Encoding", file.path(.libPaths()[1], pkg, "DESCRIPTION"))

# [SUCCEEDS]
install.packages(pkg, repos="https://cloud.r-project.org/")
# Returns Encoding="latin1"
desc::desc_get("Encoding", file.path(.libPaths()[1], pkg, "DESCRIPTION"))

[davidkim83]

I did find one workaround

*Adding INSTALL_opts="--no-html" to the install.packages(...) did the job.
Amazingly, RStudio still shows the help HTMLs, though I have no idea how this is pulled off.

+ FYI, As it turns out there are 139 packages that show this behavior as of 25 May 2023.
Test code:

LOCAL_CRAN_PATH="/path/to/your/cloned/cran/src/contrib"
LATIN1_DESCS="/path/to/your/latin1-descs.log"
find ${LOCAL_CRAN_PATH} -maxdepth 1 -type f -name "*.tar.gz" -exec tar xOzf {} */DESCRIPTION \; -exec echo ""\;\
    | grep -E "Package:|(Encoding: latin1)"\
    | grep -B 1 "Encoding: latin1"\
    2>&1 | ${LATIN1_DESCS}

[Gabor]

What is your OS and R version? FWIW I cannot reproduce this on macOS.

[davidkim83]

AlmaLinux 8.7
LANG=en_US.UTF-8
R version 4.3.0, installed via EPEL (4.3.0-1.el8)
Is there any other information that can help reproduce the issue?

[Gabor]

Yes, I can indeed reproduce it now, I'll ask the PPM (RSPM) team.

[tylfin]

Hey David,

Engineer on the Package Manager team here

Thanks for calling our attention to this issue. This was a bug with our DESCRIPTION logic that we've fixed for all new packages served by Package Manager.

We went ahead and patched the problematic packages you referenced, and those should be serving correctly as soon as the cache expires,

I hope that helps,

Tyler

[davidkim83]

Thanks for the quick action!
Will the PPM repository host binaries of those packages as well?
(provided they can indeed be compiled out-of-the-box)
+I'm not asking for their immediate provision, but it'd be good to recheck and see what can be done I think.

[davidkim83]

I can see that some packages have already been successfully built and distributed as binaries.
Thanks again