Troubleshooting GitHub tokens

I'll start with my high-level goal before dropping down into what I think is wrong-- I would like to install a package using remotes::install_github(). However, when I try to do so, I get the following error:

Using github PAT from envvar GITHUB_PAT
Error: Failed to install 'unknown package' from GitHub:
  HTTP error 401.
  Bad credentials

  Rate limit remaining: 37/60
  Rate limit reset at: 2021-09-15 17:23:41 UTC

To me, that suggests that there's something wrong with the connection between RStudio and GitHub. I tried usethis::git_sitrep() to see what was going on, and get the following results

Git config (global)
• Name: 'Amelia McNamara'
• Email: 'amelia.a.mcnamara@gmail.com'
• Vaccinated: TRUE
• Default Git protocol: 'https'
GitHub
• Default GitHub host: 'https://github.com'
• Personal access token for 'https://github.com': '<discovered>'
x Token is invalid
x Can't retrieve registered email address(es)
  If you are troubleshooting, check GitHub host, token, and token scopes
Git repo for current project
• Active usethis project: '/Users/amelia'
ℹ Active project is not a Git repo

However, I don't know what it means to "check GitHub host, token, and token scopes." I have tried generating a new token and adding it as a credential using gitcreds::gitcreds_set(), but that doesn't change the results from git_sitrep(). What should my next troubleshooting step be?

Some additional details:

  1. while my sitrep says "Can't retrieve registered email address(es)," in the GitHub section, the email address listed in the Git config section is my main GitHub email.
  2. I can use the push and pull buttons in the git pane in RStudio, as well as push and pull at the command line. My credentials seem to be cached in whatever way they need to be for that to work.

There are potentially multiple things going on here.

  1. The remotes package does not look for your credentials (your PAT) in the Git credential store the way that several other packages do. Basically, remotes uses a credential storage model from an earlier era, where we all defined our GITHUB_PAT as an env var. Therefore, for remotes to successfully find your PAT, you have to get it into the expected env var, either by directly setting it in .Renviron or by "tickling" things with another package, which will cause the PAT to be stored as an env var for the duration of the current session.
  2. The PAT built-in to the remotes package recently needed to be rolled, so one has to update remotes if you want the built-in PAT to work.
  3. GitHub changed the format of its tokens in April and it's advised (maybe now required?) to get a PAT in the new format.

gh::gh_whoami() is a good function for checking the scopes associated with your token.

Yours does seem to be invalid (versus "just" under-scoped), but it's hard to say more from here.

The email address shown in the Git config section is being read out of a local file: your user-level Git config file.

Whereas this:

"Can't retrieve registered email address(es)," in the GitHub section

says that when we send your token to GitHub, instead of getting info on who are you are (including your email address) we get a completel failure (invalid token).

How are you getting the token? Are you using usethis::create_github_token()? If not, are you at least selecting the same scopes that usethis::create_github_token() pre-selects for you (to prevent some of these problems)?

What OS are you on?

If macOS, I would probably go into the Keychain and search for "github.com" (should be Kind = "internet password"). Do you see anything weird? Like multiple passwords? Hopefully there's just one. Actually inspect the password. Is it your PAT?

gh::gh_whoami() returns

Error in gh_process_response(raw) : 
GitHub API error (401): 
Message: Bad credentials
Read more at https://docs.github.com/rest
Type .Last.error.trace to see where the error occurred

I have tried updating my credentials with usethis::create_github_token() several times, and it always appears to be successful, i.e., gitcreds::gitcreds_set() says

-> Adding new credentials...
 -> Removing credetials from cache... 
-> Done.

(as an aside, that message has a spelling error! Maybe I should submit a PR)
but then gh::gh_whoami() says the same thing.

I'm on macOS, and I only have one thing that says Kind = "internet password." I do have another entry that is "application password" but I assume that's because I have the GitHub app on my computer. When I go through the usethis::create_github_token() process, I can see that the credential in my Keychain is updated and is the new one.

What happens if you provide a freshly created PAT (created with usethis::create_github_token()) directly to gh::gh_whoami()?

We need to separate the question of "is this token good vs bad?" from "are we retrieving the token we think we are?"

I would not be surprised if this other token is a source of trouble.

I figured it out! I had some old PAT in my .Renviron, from when I was teaching about package development a while back. I updated the .Renviron to have the same PAT I had provided to gitcreds::gitcreds_set() and everything seems good now. Should I just remove the PAT from my .Renviron? Where does the credential go when you pass it to gitcreds::gitcreds_set()?

Ha! I just helped someone else with EXACTLY the same symptoms and we figured the same thing out.

I was racing over here to tell you!

Yes, definitely remove the GITHUB_PAT=lskdfjlskjflskfjlskdfj line from .Renviron.
Restart R.
Tell gitcreds again about your PAT. (Actually this may not even be necessary, but it also doesn't hurt.)
And then you should be back in business.

I am going to build something into usethis and its diagnostics (e.g. git_sitrep()) to look for this phenomenon and offer some advice.

Awesome, thank you! I do think it would be good to have in usethis diagnostics, because it sounds like I'm not the first person to encounter this issue!