Hello, I'm using the package scholar to extract the citation information based on the first name, the last name and the affiliation. However, after certain runs, it continues to return, Warning: Response code 429. Google is rate limiting you for making too many requests too quickly. Any suggestions or advice to avoid it?
Here is my code:
# Load the library
library(scholar)
# Create an empty list to store the values
scholar_id_ivy <- list()
for (i in 1:600) {
print(i) # Print the current iteration number
cat("Processing scholar", i, "\n")
id <- tryCatch({
get_scholar_id(first_name = unique_first_last_affiliation$First.Name[i],
last_name = unique_first_last_affiliation$Last.Name[i],
affiliation = unique_first_last_affiliation$University[i])
}, warning = function(w) {
cat("Warning:", conditionMessage(w), "\n")
return(NULL)
}, error = function(e) {
cat("Error:", conditionMessage(e), "\n")
return(NULL)
}, finally = {
cat("Completed processing scholar", i, "\n")
})
scholar_id_ivy[[i]] <- id
# Add a 5-second delay between requests
Sys.sleep(5)
}
I tried to use the official google scholar API but I did not find any official Google products. If you know other API or packages can handle this issue, please let me know about it. Really appreciate it.