curl -X POST --user API_TOKEN:x https://app.bluefolder.com/api/2.0/serviceRequests/list.aspx
--data "<request><listType>full</listType></request>"
to
curl command in Rcurl?
I was given an API Token that replaces API_TOKEN in the code above. I'm trying to use Rcurl to retrieve data and export it to a csv file on my shared network drive. I tested the code above in Git Bash and it worked. When I try R, I run into errors. I will use Example123 as the token example.
# devtools::install_github("hrbrmstr/curlconverter")
library(curlconverter)
#Pasting curl command line
flat <- straighten("curl -X POST --user Example123:x https://app.bluefolder.com/api/2.0/serviceRequests/list.aspx
--data '<request><listType>full</listType></request>'")
#Turns the URL parts list into a fully functional httr call. It's vectorized.
req <- make_req(flat)[[1]]
req
#Reformat the function source to make it more readable
httr::VERB(verb = "POST",
url = "https://app.bluefolder.com/api/2.0/serviceRequests/list.aspx",
httr::authenticate(user = "Example123", password = "x"),
httr::verbose(),
httr::add_headers(),
encode = "json")
#Translate that to a plain POST call without namespacing
POST(url = "https://app.bluefolder.com/api/2.0/serviceRequests/list.aspx",
authenticate(user = "Example123", password = "x"),
verbose(),
add_headers(),
encode = "json")
The error I get:
<response status='fail'>
<error code='400'>
<![CDATA[Invalid request: request is empty.]]>
</error>
I fixed the code and got a lot farther. The last thing I need to do is convert the raw data from the API call into JSON format. Then I can parse the JSON using the jsonlite package. Finally convert the parsed JSON to a data frame for analysis. I'm getting this weird lexical error on the 'fromJSON' line. Maybe an encoding issue. An xml issue. I'm not sure.
library(curlconverter)
library(httr)
library(jsonlite)
#Pasting curl command line
flat <- straighten("curl -X POST --user Example123:x https://app.bluefolder.com/api/2.0/customers/list.aspx --data '<request><customerList><listType></listType></customerList></request>'")
#Turns the URL parts list into a fully functional httr call. It's vectorized.
req <- make_req(flat)[[1]]
#Reformat the function source to make it more readable
httr::VERB(verb = "POST",
url = "https://app.bluefolder.com/api/2.0/customers/list.aspx",
body = "<request><customerList><listType></listType></customerList></request>",
httr::authenticate(user = "Example123", password = "x"),
httr::verbose(),
httr::add_headers(),
encode = "json")
#Translate that to a plain POST call without namespacing
getInfoInJson <- POST(url = "https://app.bluefolder.com/api/2.0/customers/list.aspx",
body = "<request><customerList><listType></listType></customerList></request>",
authenticate(user = "Example123", password = "x"),
verbose(),
add_headers(),
encode = "json")
curl_string <- "curl -X POST --user Example123:x https://app.bluefolder.com/api/2.0/customers/list.aspx --data '<request><customerList><listType></listType></customerList></request>'"
make_req(straighten(curl_string))[[1]]()
#accesses the body as a character vector
get_prices_text <- content(getInfoInJson, "text", encoding = "UTF-8")
#converts the raw data from API call into JSON format. Then can parse the JSON using the jsonlite package
#Getting errors from this line below
get_prices_json <- fromJSON(get_prices_text, flatten = TRUE)
#convert the parsed JSON to a data frame for analysis
get_prices_df <- as.data.frame(get_prices_json)
you're welcome. Maybe the response you have is xml and not json, I took a quick look at their documentation, and didn't find how to get the json response.
So you can use xml2 to query the content(getInfoInJson) using XPath and then create your dataframe.
For example:
cName = content(getInfoInJson) %>% xml2::xml_find_all("//customerName") %>% xml2::xml_text()
Also, if you know how to formulate the query, you don't need to use most of the commands you have there, I think this part is enough:
Perfect!! It worked man. Thank you so much! I got the list of customers in a nice format. I just have one more question, then I'm done. In the documentation, there's a response section. I want to include customerId, customerName, customerType, inactive, and externalId to the dataframe. Can I paste what's in the documentation in the "body =" field for the POST function? Or do I include it on another line? I didn't see headers in the csv but I'm assuming I can make them with the add_headers() function.
There's one xml file with invalid characters in them. The file has "&", ">", "'", """ . I have the code below and the error comes at the serviceXML = content(getInfoInJson) part. Do you know the smartest way to get around these characters?
> cName <- content(getInfoInJson)
Error in read_xml.raw(x, encoding = encoding, ...) :
xmlParseCharRef: invalid xmlChar value 11 [9]
For assignment, I tried to get the assignedTo field but there's something nested inside it (userId).
I've tried using the code below but it has a different number of rows compared to all the other fields and I get an error. Is there a proper way to code the below?
Hi, for (1) I don't know, can you provide an example of the xml response that doesn't work?
(2): it depends on the structure of the XML file, this is an example of how to do it based on the response provided in the documentation:
After making getInfoJson, is there a way to replace the special characters in this xml variable? Read_xml isn't working after it's made. I don't think an xml response is the reason. It's the special characters inside the getInfoJson.