How to specify user agent when using github API?

[bretauv]

I want to obtain the list of files on a Github repo. I followed this answer but I noticed that sometimes I have a HTTP 403 error. For example, if I run the following code:

library(httr)

for (i in 1:10) {
  req <- GET("https://api.github.com/repos/etiennebacher/tidytuesday/git/trees/master?recursive=1")
  stop_for_status(req)
}

> Error: Forbidden (HTTP 403).

(Note that I don't actually want to run this GET request 10 times, it's just the easiest way I found to simulate my problem. )

Searching a bit online, I found this answer that explains that the Github API requires the GitHub username, or the name of the application, for the User-Agent header value of the request.

But adding user_agent("etiennebacher") in GET() doesn't change anything. How should I specify the user agent in this case?

Also asked on StackOverflow

[bretauv]

Answer on StackOverflow:

As commented by @MrFlick, the HTTP 403 here means that the API rate limit was exceeded. A way to solve this is to authenticate when making the request, because the rate limit goes from 60 to 5,000 requests per hour. It is easier to do so by using the package gh (and its eponymous function). This can be done locally by specifying .token = <PAT> (where <PAT> is your GitHub Personal Access Token) in gh().

To obtain the list of files in a particular repo, you can save the output of gh() in a JSON file, and then read it with jsonlite::fromJSON(). It is then trivial to get the list of files.

Bottom line, this works:

library(gh)
library(jsonlite)

tmp_file <- tempfile(fileext = ".json")

gh::gh("https://api.github.com/repos/etiennebacher/tidytuesday/git/trees/master?recursive=1", .destfile = tmp_file, .token = <PAT>)

jsonlite::fromJSON(tmp_file)$tree$path

Bonus: how to authenticate in Github Actions

I didn't say it in the original post, but this GET request was supposed to be made in a Github action. Since you can't manually provide the PAT as a token, you need to define the token in the .yaml file, and then pass its value in the .R file.

test.R

library(gh)
library(jsonlite)

tmp_file <- tempfile(fileext = ".json")

gh::gh("https://api.github.com/repos/etiennebacher/tidytuesday/git/trees/master?recursive=1", .destfile = tmp_file, .token = github_token)

jsonlite::fromJSON(tmp_file)$tree$path

GitHub action:

on:
  push:
    branches: master

jobs:
  build:
    runs-on: macOS-latest
    steps:
      - uses: actions/checkout@v2
      - uses: r-lib/actions/setup-r@master
      - run: |
          github_token <- "${{ secrets.GITHUB_TOKEN }}"
          install.packages(c("gh", "jsonlite"))
          source("test.R"),
        shell: Rscript {0}