Vetiver problem with docker

Disclaimer: I know almost nothing of docker.

I have the following vetiver generated file of my model, so pretty standard:

Generated by the vetiver package; edit with care

FROM rocker/r-ver:4.3.1
ENV RENV_CONFIG_REPOS_OVERRIDE https://packagemanager.rstudio.com/cran/latest

RUN apt-get update -qq && apt-get install -y --no-install-recommends
libcurl4-openssl-dev
libicu-dev
libsodium-dev
libssl-dev
make
zlib1g-dev
&& apt-get clean

COPY vetiver_renv.lock renv.lock
RUN Rscript -e "install.packages('renv')"
RUN Rscript -e "renv::restore()"
COPY plumber.R /opt/ml/plumber.R
EXPOSE 8080
ENTRYPOINT ["R", "-e", "pr <- plumber::plumb('/opt/ml/plumber.R'); pr$run(host = '0.0.0.0', port = 8080)"]

But no matter if I put it on huggingface or render, I get:

error: failed to solve: process "/bin/sh -c Rscript -e \"renv::restore()\"" did not complete successfully: exit code: 1
error: exit status 1

even though I followed Use Docker to deploy a model for #TidyTuesday LEGO sets | Julia Silge , the official documentation, a video. GPT proved not useful so I feel clueless on what to do. Do you have suggestions?

Oh, that sounds frustrating!

I wonder if something is wrong/wonky with your renv.lock file.

  • What does your renv.lock look like? Pretty normal?
  • What happens if you go to the directory where you have the renv.lock and you try to run Rscript -e "renv::restore()" outside of Docker, just from the command line. Is renv able to restore that set of packages? This isn't a perfectly analogous thing to do because within Docker, you are working on a little Linux machine while you may be using a Windows or macOS machine; the package installation situation is pretty different across these operating systems.
  • Unfortunately I can't say it, I'd say yes. It looks pretty big. First lines
{
  "R": {
    "Version": "4.3.1",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "http://rspm/default/latest"
      }
    ]
  },
  "Packages": {
    "R6": {
      "Package": "R6",
      "Version": "2.5.1",
      "Source": "Repository",
      "Repository": "RSPM",
      "Requirements": [
        "R"
      ],
      "Hash": "470851b6d5d0ac559e9d01bb352b4021"
    },
    "Rcpp": {
      "Package": "Rcpp",
      "Version": "1.0.11",
      "Source": "Repository",
      "Repository": "RSPM",
      "Requirements": [
        "methods",
        "utils"
      ],
      "Hash": "ae6cbbe1492f4de79c45fce06f967ce8"
    },
  • If I go to cmd, if I understood you, and type Rscript -e "renv::restore()", inside the folder of where I have all, I get an error that indicates I don't know what I just did

That renv.lock looks about right, so I expect there isn't a problem with that.

Maybe you can say a little more about what happens when you try to do renv::restore()? If you are having trouble with it from the command line, try first from within RStudio. You'd make a new project/folder (don't do this in any of your "real" projects), add this renv.lock file to the project, and then try renv::restore() in this new project. It will try to install all the packages with these specific versions into that new project. If that succeeds, then you know there isn't a problem with the renv.lock file itself.

I created a new project with renv.
I deleted the .lock file it created, I pasted the vetiver_renv.lock
I ran

* Project '~/GitHub/Renv test' loaded. [renv 0.15.5]
> renv::restore()
* The library is already synchronized with the lockfile.
>

I don't feel sure this answers

Thanks with your patience on my response! It sounds like the renv.lock file is in fact fine, which makes sense.

  • Can you build more simple Docker containers that run R? For example, can you successfully work through this tutorial?

  • Can you try building a Docker container with renv, but for a more simple application than serving a model? For example, can you go back to that more simple tutorial and switch out the package installation from install.packages() to using renv? I find this and this helpful for using renv together with Docker.

You also may want to work through these resources listed in the vetiver docs.

I'll take a week or so to go through them, thanks

In the code you provide here: MLOps with vetiver - Deploy with Docker

I just modiefied

board <- board_folder(path = "pins-r", versioned = TRUE)
vetiver_pin_write(board, v)
vetiver_prepare_docker(board, "superbowl_rf")

But I coulnd't find a solution for

#ERROR: "docker buildx build" requires exactly 1 argument.

After I ran

docker build -t superbowlads

on powershell

I don't feel sure this belongs to this topic

Solved with: dockerfile - Docker build "path . not found" - Stack Overflow
in particular

docker build -t superbowlads C:\Users\Name\Downloads

1 Like

Ah, interesting! So Docker couldn't find your Dockerfile, even if you were in the directory where it was? Thanks for sharing what helped you figure it out. :+1:

New day, new problem. When running the container in docker, I got an error that weirded me out. It showed a model not used in the superbowlads dockerfile. This because I put the dockerfile inside Downloads, where previously I put a different vetiver_renv.lock and plumber.R file, of the first attempt that lead me to write this post. When running the new container I got:

2023-12-19 15:16:43 Error in stopOnLine(lineNum, file[lineNum], e) :
2023-12-19 15:16:43 Error on line #7: 'b <- board_folder(path = "pins-r")' - Error in pin_meta():
2023-12-19 15:16:43 ! Can't find pin called "superbowl_rf"

but in the folder pins-r I see superbowl_rf with the versions

I guess the problem comes from the warning

We don’t recommend that you store your model inside your container, but (if appropriate to your use case) it is possible to edit the generated Dockerfile and COPY the folder and model into the container. Alternatively, you can mount the folder as a VOLUME.

Yes. Using board_folder gives problems. So I asked ChatGPT a solution:

Generated by the vetiver package; edit with care

FROM rocker/r-ver:4.3.2
ENV RENV_CONFIG_REPOS_OVERRIDE https://packagemanager.rstudio.com/cran/latest

RUN apt-get update -qq && apt-get install -y --no-install-recommends
libcurl4-openssl-dev
libicu-dev
libsodium-dev
libssl-dev
make
zlib1g-dev
&& apt-get clean

Create a directory to store the pins-r folder

WORKDIR /opt/ml
RUN mkdir pins-r

Copy the local "pins-r" folder into the container

ADD pins-r /opt/ml/pins-r

COPY vetiver_renv.lock renv.lock
RUN Rscript -e "install.packages('renv')"
RUN Rscript -e "renv::restore()"
COPY plumber.R /opt/ml/plumber.R
EXPOSE 8000
ENTRYPOINT ["R", "-e", "pr <- plumber::plumb('/opt/ml/plumber.R'); pr$run(host = '0.0.0.0', port = 8000)"]

This works. But now I can't access

Running plumber API at http://0.0.0.0:8000

I also tried using the non universal IP

Hi,

I worked through this and got it working in a similar fashion. Try these updates:

Dockerfile

RUN mkdir /opt/ml/pins-r
ADD pins-r/ /opt/ml/pins-r

instead of

WORKDIR /opt/ml
RUN mkdir pins-r
ADD pins-r /opt/ml/pins-r

plumber.R

b <- board_folder(path = "/opt/ml/pins-r")

For me, going to http://127.0.0.1:8000/__docs__/ then loaded:

If I change the dockerfile and the plumber, I get

0.361 mkdir: cannot create directory ‘/opt/ml/pins-r’: No such file or directory

if I just change the plumber or if I change the plumber and the dockerfile using

WORKDIR /opt/ml
RUN mkdir /opt/ml/pins-r
ADD pins-r/ /opt/ml/pins-r

same situation I had in the previous reply

Then in that case, what happens when you try:
RUN mkdir -p /opt/ml/pins-r
Without changing the working directory and keeping plumber.R as
b <- board_folder(path = "/opt/ml/pins-r")

If I understood, instead of

WORKDIR /opt/ml
RUN mkdir /opt/ml/pins-r
ADD pins-r/ /opt/ml/pins-r

I put only

RUN mkdir -p /opt/ml/pins-r

I got

2023-12-22 08:41:53 Error in stopOnLine(lineNum, file[lineNum], e) :
2023-12-22 08:41:53 Error on line #7: 'b <- board_folder(path = "/opt/ml/pins-r")' - Error in pin_meta():
2023-12-22 08:41:53 ! Can't find pin called "superbowl_rf"

If I use

WORKDIR /opt/ml
RUN mkdir -p /opt/ml/pins-r
ADD pins-r/ /opt/ml/pins-r

same problem of the previous image

Ah right. So I think an explanation would help. The docker image is basically the OS in a file that someone created. I don't know what is needed in the default entry area, so I do not recommend changing the WORKDIR unless you change it back. Otherwise, the container might not have access to other files it needs that were built in.

To make sure the full file directory was created, we want to use
RUN mkdir -p /opt/ml/pins-r
without -p it would error out (like you saw here mkdir: cannot create directory ‘/opt/ml/pins-r’: No such file or directory) if the /opt/ml directory was not present. With the -p it will create the parent directories if needed.

And then ADD pins-r/ /opt/ml/pins-r is going to make your local pins-r/ directory available within the docker container at the location /opt/ml/pins-r.

So I think your Dockerfile should look like this:

FROM rocker/r-ver:4.3.2
ENV RENV_CONFIG_REPOS_OVERRIDE https://packagemanager.rstudio.com/cran/latest

RUN apt-get update -qq && apt-get install -y --no-install-recommends
libcurl4-openssl-dev
libicu-dev
libsodium-dev
libssl-dev
make
zlib1g-dev
&& apt-get clean

RUN mkdir -p /opt/ml/pins-r
ADD pins-r/ /opt/ml/pins-r

COPY vetiver_renv.lock renv.lock
RUN Rscript -e "install.packages('renv')"
RUN Rscript -e "renv::restore()"
COPY plumber.R /opt/ml/plumber.R
EXPOSE 8000
ENTRYPOINT ["R", "-e", "pr <- plumber::plumb('/opt/ml/plumber.R'); pr$run(host = '0.0.0.0', port = 8000)"]

And then in the plumber.R file, I added the full path:
b <- board_folder(path = "/opt/ml/board")
but that might not be necessary since the ENTRYPOINTpoints to the plumber.R file in that directory. So try both that and
b <- board_folder(path = "board")
and hopefully with that Dockerfile one will work.

This looks very weird

dockerfile:5

3 |
4 | RUN apt-get update -qq && apt-get install -y --no-install-recommends
5 | >>> libcurl4-openssl-dev
6 | libicu-dev
7 | libsodium-dev

ERROR: failed to solve: dockerfile parse error on line 5: unknown instruction: libcurl4-openssl-dev

Even if I run the non edited version. Trying to update Docker...

Yeah, looks like the slashes got formatted out at some point. That part should be:

RUN apt-get update -qq && apt-get install -y --no-install-recommends \
  libcurl4-openssl-dev \
  libicu-dev \
  libsodium-dev \
  libssl-dev \
  make \
  zlib1g-dev \
  && apt-get clean