call_geolocator_latlon curl::curl_fetch_memory(url, handle = handle) : URL using bad/illegal format or missing URL error

[jillahmad17]

Hi all,

I am calling the call_geolocator_latlon function from the tigris package in order to convert latitude and longitude points into census tracts.

I have a dataset with 350,000 rows. Initially, the code runs for first 3,000 rows:

## dummy variable
FY23seekingAssist$tracts <- 0
FY23seekingAssist[1:3000,]$tracts <- apply(FY23seekingAssist[1:3000,], 1,
                                     function(row) call_geolocator_latlon(row['Latitude'], row['Longitude'], ))

However, for the next 3,000 rows, as shown below, I get this error:

## dummy variable
FY23seekingAssist$tracts <- 0
FY23seekingAssist[3001:6000,]$tracts <- apply(FY23seekingAssist[3001:6000,], 1,
                                     function(row) call_geolocator_latlon(row['Latitude'], row['Longitude'], ))

Error in curl::curl_fetch_memory(url, handle = handle) :
URL using bad/illegal format or missing URL

Does anyone have any insight into what is causing this error? Another question I have is how do I most effectively run this code for 350,000 rows? I don't want to have to change the rows within the function for every row but will do so if that is what it takes....

Thank you so much in advance.

PS I would post a snippet of the dataset but the information is confidential.

[technocrat]

I haven’t been able to find much on this error. Since there are far fewer Census tracts that the number of coords you are trying to match ( 85,528 vs 350,000) you might be better of fetching the sf geometries and then locally matching the centroids and your pairs.

[jillahmad17]

That is really helpful; thank you for looking into this. Do you mind clarifying what you mean by matching the centroids? I understand turning the lat and long points into sf objects (geom_point) but not matching the centroids and pairs. Appreciate the insight.

[technocrat]

I was confused, thinking that it was distance from point to tract. Here's what I think you need instead

library(sf)
#> Linking to GEOS 3.11.0, GDAL 3.5.3, PROJ 9.1.0; sf_use_s2() is TRUE
library(tigris)
#> Warning: package 'tigris' was built under R version 4.3.1
#> To enable caching of data, set `options(tigris_use_cache = TRUE)`
#> in your R script or .Rprofile.
options(tigris_use_cache = TRUE)
nc_tracts <- tracts("NC",
                    cb = TRUE,
                    year = 2022)
dim(nc_tracts)
#> [1] 2660   14

nc_tracts[1,]
#> Simple feature collection with 1 feature and 13 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: -79.40238 ymin: 35.51743 xmax: -79.08809 ymax: 35.68678
#> Geodetic CRS:  NAD83
#>   STATEFP COUNTYFP TRACTCE             AFFGEOID       GEOID NAME
#> 1      37      037  020600 1400000US37037020600 37037020600  206
#>           NAMELSAD STUSPS     NAMELSADCO     STATE_NAME LSAD     ALAND  AWATER
#> 1 Census Tract 206     NC Chatham County North Carolina   CT 303405835 2361346
#>                         geometry
#> 1 MULTIPOLYGON (((-79.40216 3...
# some points to use for matching since we know they are

my_points <- data.frame(
  longitude = c(-81.00000,-78.47397, -80.18244, -83.13667, -80.24383, -80.76486),
  latitude = c(37.00000,35.73328, 36.21689, 35.66299, 36.0188, 35.53215)
)

points_sf <- st_as_sf(my_points, coords = c("longitude", "latitude"), crs = 4269)
the_matches <- st_join(points_sf,nc_tracts)
the_matches[,c("NAMELSAD","geometry")]
#> Simple feature collection with 6 features and 1 field
#> Geometry type: POINT
#> Dimension:     XY
#> Bounding box:  xmin: -83.13667 ymin: 35.53215 xmax: -78.47397 ymax: 37
#> Geodetic CRS:  NAD83
#>              NAMELSAD                   geometry
#> 1                <NA>             POINT (-81 37)
#> 2 Census Tract 541.09 POINT (-78.47397 35.73328)
#> 3  Census Tract 29.04 POINT (-80.18244 36.21689)
#> 4   Census Tract 9801 POINT (-83.13667 35.66299)
#> 5     Census Tract 36  POINT (-80.24383 36.0188)
#> 6 Census Tract 614.08 POINT (-80.76486 35.53215)

^{Created on 2023-10-16 with reprex v2.0.2}

[jillahmad17]

This is so helpful; thank you for the insight!!