How do I iterate through a dataframe to getdeepsearch results in zillow api?

iheart · July 15, 2019, 4:53pm

Hi, Im trying to work with the zillow api for the first time:

I was able to find my desired output for one address if I manually typed it:

library(XML)
library(ZillowR)
set_zillow_web_service_id("X1-ZWz17seirkzuh7_93aho")

xml <- GetDeepSearchResults(address='600 S. Quail Ct.', citystatezip ='67114',
rentzestimate=FALSE)

dat <- unlist(xml)
str(dat)

dat <- as.data.frame(dat)
dat <- gsub("text","", dat$dat)

head(dat, 20)

I have a dataset(house) in csv which has a list of all necessary parameters like state,zip,address.

Repex:
head(house)

  AccountId                    Street        City State ZipCode abcFactorPct swfFactorPct ZipHhIncome ZipPctAdvDegree
1    465xyz            5b9 Ventura Ct Suisun City    CA   94585            0.8         2.75       70491      0.04138845
2    4505ab 20235 North Cave Creek Rd     Phoenix    AZ   85024            0.9         2.50       68557      0.12165522
3    4854lm           371 50th Street     Oakland    CA   94609            0.8         2.75       67827      0.22325981

I want to get information for all rows of data in this dataframe. How can I do the same in R?

andresrcs · July 15, 2019, 7:03pm

This is one way to do it with tidyverse functions.

library(ZillowR)
library(tidyverse)

# Sample data
house <- data.frame(stringsAsFactors=FALSE,
                 AccountId = c("465xyz", "4505ab", "4854lm"),
                 Street = c("5b9 Ventura Ct", "20235 North Cave Creek Rd",
                            "371 50th Street"),
                 City = c("Suisun City", "Phoenix", "Oakland"),
                 State = c("CA", "AZ", "CA"),
                 ZipCode = c(94585, 85024, 94609),
                 abcFactorPct = c(0.8, 0.9, 0.8),
                 swfFactorPct = c(2.75, 2.5, 2.75),
                 ZipHhIncome = c(70491, 68557, 67827),
                 ZipPctAdvDegree = c(0.04138845, 0.12165522, 0.22325981)
)

set_zillow_web_service_id("X1-ZWz17seirkzuh7_93aho")

map2_dfr(house$Street, house$ZipCode,
        ~{GetDeepSearchResults(address = .x,
                               citystatezip = as.character(.y),
                               rentzestimate=FALSE) %>% 
            unlist() %>% 
            enframe()
          }) %>% 
  filter(!str_detect(value, "text"))
#> # A tibble: 167 x 2
#>    name                          value                                     
#>    <chr>                         <chr>                                     
#>  1 request.address               5b9 Ventura Ct                            
#>  2 request.citystatezip          94585                                     
#>  3 message.text                  Error: no exact match found for input add…
#>  4 message.code                  508                                       
#>  5 request.address               20235 North Cave Creek Rd                 
#>  6 request.citystatezip          85024                                     
#>  7 message.text                  Request successfully processed            
#>  8 message.code                  0                                         
#>  9 response.name                 response                                  
#> 10 response.children.results.na… results                                   
#> # … with 157 more rows

^{Created on 2019-07-15 by the reprex package (v0.3.0)}

iheart · July 15, 2019, 7:24pm

@andresrcs my dataset(csv) is larger than this repex about 5k rows. My questions was ow can it iterate through the entire dataset without manually typing each parameter? Basically how do I automate it to pick up all the parameters to hit the getdeepsearcgresults api and give me my final output for each address

andresrcs · July 15, 2019, 7:38pm

I already gave you the answer, there is no need for typing, you just have to replace the sample data (that you provided) with your actual dataframe.

iheart · July 15, 2019, 7:39pm

@andresrcs ok I did the same thank you but I received an error Error in GetDeepSearchResults(address = .x, citystatezip = as.character(.y), :
'address' must be character

andresrcs · July 15, 2019, 7:41pm

I don't really have enough info to help you out with that. Could you ask this with a minimal REPRoducible EXample (reprex)? A reprex makes it much easier for others to understand your issue and figure out how to help.

If you've never heard of a reprex before, you might want to start by reading this FAQ:

FAQ: How to do a minimal reproducible example ( reprex ) for beginners Guides & FAQs

A minimal reproducible example consists of the following items: A minimal dataset, necessary to reproduce the issue The minimal runnable code necessary to reproduce the issue, which can be run on the given dataset, and including the necessary information on the used packages. Let's quickly go over each one of these with examples: Minimal Dataset (Sample Data) You need to provide a data frame that is small enough to be (reasonably) pasted on a post, but big enough to reproduce your issue. Let's say, as an example, that you are working with the iris data frame head(iris) #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species #> 1 5.1 3.5 1.4 0.…

iheart · July 15, 2019, 7:42pm

Can you please comment why I am receiving error - Error in GetDeepSearchResults(address = .x, citystatezip = as.character(.y), :
'address' must be character

andresrcs · July 15, 2019, 7:44pm

Again, I don't have enough information, because I'm not seeing the same data as you, maybe you have read your csv file as factors instead of character strings but I can't be sure.

iheart · July 15, 2019, 7:45pm

ok can you please tell me what the syntax .x and .y is

andresrcs · July 15, 2019, 7:50pm

Those are the names of map2_dfr() arguments and in this particular situation they are set like this

.x = house$Street
.y = house$ZipCode

andresrcs · July 15, 2019, 8:01pm

Have you read the link I gave you about how to make a reprex?
What you are posting is not a reprex, not even sample data, is a screenshot and it is not a good thing to do here.

iheart · July 15, 2019, 8:14pm

Hi @andresrcs

Please find below reprex of the data using datapasta. @andresrcs this is correct reprex acc to the link. Would really appreciate your help.

tibble::tribble(
  ~AccountId, ~ZipCode, ~TotalAllowed, ~ZipHhIncome,                ~Street, ~State, xxxxxxxxxxxxPct, ~ZipPctAdvDegree, ~AgeFactorPct,             ~City,
      464641,    43130,             0,        46205,    "2577 Long Bow Ave",   "OH",             0.8,      .058499952,           0.7,       "Lancaster",
      451304,    44718,             0,        69396,    "4822 Armandale Nw",   "OH",             0.8,      .171958907,           0.7,          "Canton",
      443925,     8837,             0,        74764,    "348 Grandview Ave",   "NJ",               1,       0.16989858,           0.7,          "Edison",
      464725,     2032,           147,       100658,       "81 Rhoades Ave",   "MA",             1.3,      0.247449819,           0.7,    "East Walpole",
      431671,    89403,        335.86,        55248, "296 Monte Cristo Dr.",   "NV",             0.8,      0.066260613,           0.7,          "Dayton",
      474844,    99703,         61.05,        53031,      "4348 9th Street",   "AK",            1.05,      0.061620898,           0.8, "Fort Wainwright",
      440990,    55429,             0,        43835, "5649 Vera Cruz Ave N",   "MN",               1,      0.050472833,           0.7,         "Crystal"
  )

andresrcs · July 15, 2019, 9:37pm

Well, that is sample data, not a reprex, if I use this new sample data, I still can't reproduce your issue, it works as expected for me, see reprex below:

house <- tibble::tribble(
  ~AccountId, ~ZipCode, ~TotalAllowed, ~ZipHhIncome,                ~Street, ~State, ~xxxxxxxxxxxxPct, ~ZipPctAdvDegree, ~AgeFactorPct,             ~City,
  464641,    43130,             0,        46205,    "2577 Long Bow Ave",   "OH",             0.8,      .058499952,           0.7,       "Lancaster",
  451304,    44718,             0,        69396,    "4822 Armandale Nw",   "OH",             0.8,      .171958907,           0.7,          "Canton",
  443925,     8837,             0,        74764,    "348 Grandview Ave",   "NJ",               1,       0.16989858,           0.7,          "Edison",
  464725,     2032,           147,       100658,       "81 Rhoades Ave",   "MA",             1.3,      0.247449819,           0.7,    "East Walpole",
  431671,    89403,        335.86,        55248, "296 Monte Cristo Dr.",   "NV",             0.8,      0.066260613,           0.7,          "Dayton",
  474844,    99703,         61.05,        53031,      "4348 9th Street",   "AK",            1.05,      0.061620898,           0.8, "Fort Wainwright",
  440990,    55429,             0,        43835, "5649 Vera Cruz Ave N",   "MN",               1,      0.050472833,           0.7,         "Crystal"
)

library(ZillowR)
library(tidyverse)


set_zillow_web_service_id("X1-ZWz17seirkzuh7_93aho")

map2_dfr(house$Street, house$ZipCode,
         ~{GetDeepSearchResults(address = .x,
                                citystatezip = as.character(.y),
                                rentzestimate=FALSE) %>% 
             unlist() %>% 
             enframe()
         }) %>% 
  filter(!str_detect(value, "text"))
#> # A tibble: 534 x 2
#>    name                                               value                
#>    <chr>                                              <chr>                
#>  1 request.address                                    2577 Long Bow Ave    
#>  2 request.citystatezip                               43130                
#>  3 message.text                                       Request successfully…
#>  4 message.code                                       0                    
#>  5 response.name                                      response             
#>  6 response.children.results.name                     results              
#>  7 response.children.results.children.result.name     result               
#>  8 response.children.results.children.result.childre… zpid                 
#>  9 response.children.results.children.result.childre… 61456590             
#> 10 response.children.results.children.result.childre… links                
#> # … with 524 more rows

Try sharing the sample data using dput() instead of datapasta (as explained in the reprex guide) that way we could take a look to the structure of your actual dataframe.

system · August 5, 2019, 9:41pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.