R Shiny 1.7.3 package is not discoverable on R server on ubuntu 20

Continuing the discussion from How to install shiny server 1.7.1 on ubuntu 20.04:

I have the same question, but now it applies to 1.7.3.
We know that the latest version of shiny is 1.7.3 as from package news here.

But when I click on the Update button in Rstudio the listed packages donot show Shiny. My current Shiny R package is 1.7.1. So why is version 1.7.3 not available in the Rstudio package update list?

Here is the screenshot of Rstudio Update window.

Screenshot 2022-11-10 at 8.05.32 PM

...

RStudio simply shows the updates available on the package repository you have configured. Can you show what repository you have set?

getOption("repos")
 getOption("repos")

CRAN 
"https://cloud.r-project.org" 

That is weird, the reported version I get from "https://cloud.r-project.org" is 1.7.3

library(dplyr)

available.packages(contrib.url("https://cloud.r-project.org")) |>
    as_tibble() %>% 
    filter(Package == 'shiny')
#> # A tibble: 1 × 17
#>   Package Version Priority Depends       Imports Linki…¹ Sugge…² Enhan…³ License
#>   <chr>   <chr>   <chr>    <chr>         <chr>   <chr>   <chr>   <chr>   <chr>  
#> 1 shiny   1.7.3   <NA>     R (>= 3.0.2)… "utils… <NA>    "datas… <NA>    GPL-3 …
#> # … with 8 more variables: License_is_FOSS <chr>, License_restricts_use <chr>,
#> #   OS_type <chr>, Archs <chr>, MD5sum <chr>, NeedsCompilation <chr>,
#> #   File <chr>, Repository <chr>, and abbreviated variable names ¹​LinkingTo,
#> #   ²​Suggests, ³​Enhances

Created on 2022-11-10 with reprex v2.0.2

Are you sure you don't have shiny 1.7.3 installed on any of your package libraries?

Ah, I see some light.
When I run the above command, I also see that Shiny 1.7.3 on the respository.

But it is a bit worrying that the rstudio package window still shows the shiny version as 1.7.1.
I can see a possible connection with.libPaths() as it shows 4 library paths.

.libPaths()
[1] "/home/sanjay/R/x86_64-pc-linux-gnu-library/4.2" "/usr/local/lib/R/site-library"                 
[3] "/usr/lib/R/site-library"                        "/usr/lib/R/library"             

The first of the libPath is the one in which Shiny 1.7.3 is installed.
Which means rstudio is not pointing to the correct library path.
How do we check & correct that?

Can you explain why you think that is the case? the first lib path is the one you have configured to be the default one so RStudio is simply using your default setting. Have in mind that this option is set per user, not globally.

I just discovered that rstudio is pointing to the wrong libpath.

When I click on the package checkbox after unchecking it, it executed the following command on the console.

library(shiny, lib.loc = "/usr/lib/R/site-library")

The lib.loc is the old library. I guess after I installed R 4.1.2 I had to reinstall all packages, that is why there a new library. This happened earlier too when I moved from R 3.2 (if I remember correctly), causing so many lib paths to coexist. I knew this will someday cause a problem for me :frowning:

What would be the best way forward? I donot need the old libpaths now.

The "packages" panel lists all available packages grouped by lib path, very likely if you scroll you will find the same package repeated.

It seems you have installed R both from CRAN and RStudio, they install on different paths. You can delete the one you are not using, ideally, only base R packages should be kept on the R_HOME/library path, system-wide required packages on the R_HOME/site-library path, and user-specific packages on the user-level path HOME/R/x86_64-pc-linux-gnu-library/4.2.

If you are the only user on that particular server or if you use that server for production, I recommend you install all non-base packages in the site-library folder and do not use the user-level one.

Got it. We are very close to fixing this issue. Your diagnosis has been going very well. Please hold on a bit.

So I found two groups of packages, as you have said.
User Library and System Library.


And there are 4 paths.

> .libPaths()
[1] "/home/sanjay/R/x86_64-pc-linux-gnu-library/4.2"
[2] "/usr/local/lib/R/site-library"                 
[3] "/usr/lib/R/site-library"                       
[4] "/usr/lib/R/library"    

I did some digging using installed.packages() and here are two results.
Checking total packages in each of the 4 libpaths,
1:4 |> map_dbl(~installed.packages(lib.loc = .libPaths()[.x]) |> nrow() )

outputs

[1] 162 41 160 29

This shows that paths 1 & 3 have almost a half / half split of the 320 odd packages. This looks like a problem to me.

Further investigating the intersecting package names across the 4 libraries,

packnames <- 1:4 |> map(~installed.packages(lib.loc = .libPaths()[.x])[,1] )

 intersect(packnames[[1]],packnames[[2]])
[1] "hardhat" "parsnip" "xts"    
> intersect(packnames[[2]],packnames[[3]])
character(0)
> intersect(packnames[[3]],packnames[[4]])
character(0)
> intersect(packnames[[1]],packnames[[4]])
[1] "Matrix"
> intersect(packnames[[2]],packnames[[4]])
character(0)
> intersect(packnames[[1]],packnames[[3]])
[1] "callr"       "fontawesome" "htmltools"   "pillar"      "processx"    "ps"          "Rcpp"        "rlang"       "tibble"     
> 

So there are very few overlaps.

Checking the distribution of shiny related packages

> packnames[[1]] |> str_subset("shiny")
[1] "shinyalert"         "shinyBS"            "shinycssloaders"    "shinydashboardPlus" "shinyTime"         
[6] "shinyWidgets"      
> 
> packnames[[2]] |> str_subset("shiny")
character(0)
> 
> packnames[[3]] |> str_subset("shiny")
[1] "shiny"          "shinydashboard" "shinyjs"       
> 
> packnames[[4]] |> str_subset("shiny")
character(0)
> 

This shows that 3 packages are in /usr/lib/R/site-library while all other shiny related packages are in /home/sanjay/R/x86_64-pc-linux-gnu-library/4.2

Do you think I should just live with this, or move back the shiny packages from /usr/lib/R/site-library to /home/sanjay/R/x86_64-pc-linux-gnu-library/4.2?

What should be my main action now?

There is no one approach to rule them all, it depends on the actual applicantion your server is being used for. As I said, for a production setting I would keep all non base packages on the site-library folder. Have in mind that for shiny-server, the "run as" user needs to have read and execute permissions on the R packages.

Is there any easy way to delete packages from one libpath if they are already present in another. I tried uninstall button on the rstudio packages pane, and it uninstalled from both libpaths.