-
Notifications
You must be signed in to change notification settings - Fork 0
Tip Sheet: Geocoding
If you're mapping points, you need latitude and longitude coordinates. Geocoding is the process of matching individual addresses to latitude/longitude points that can be mapped. If you're mapping shapes (states, counties, countries), you should take a look at the Shapefiles tip sheet.
This is a community resource. Edits are welcome! If you favor a resource that isn't shown here or find that the terms or availability of something here has changed, by all means, help keep this resource up to date.
Geocodio offers 2,500 free lookups per day and then charges 50 cents per thousand after that. ProPublica has a write up of their tests of Geocodio and a few others. There's even a Geocodio R package.
OpenCage also has a limit of 2,500 lookups per day, a decent API and pretty forgiving limits on the paid tiers.
Pelias requires some IT to set it up, but once you've done that you have only infrastructure limitations - no license fees or rate limits. And it's very well-documented.
R's ggmap package includes geocoding. There's a nice walk through below if you want a usage example.
The Census will take batches of 1000 at a time. There's a nice walk through below if you want a usage example.
Datawrapper will do small batches for free (tk: how many?), which is a great option if you're using their tools to make a map display.
Don't want to DIY? Depending on the project, it may be worth reaching out to the UC Berkeley Cartography and GIS Education (CAGE) Lab, to see if they will do the geocoding for you. Your local university may have similar programs. note: feel free to add suggestions
Below are suggestions that should be tested out but might work for your needs. If you, or someone you love, uses one of these, please feel free to add information on limitations, to help folks decide on the best option for them.
- Mike Stucka's command line geocoder uses Google's engine and Python 2.
- Texas A&M offers a free service tk: limitations?.
- Luke Peterson wrote an excel module that he uses.
- In NYC GOAT, which is maintained by the Department of City Planning, is worth taking a look at.
- I have no idea who is behind Find Latitude and Longitude but it does a great job with small batches for free.
- The PostGIS Tiger geocodershould be unaffected by limits, but it's unclear to me whether it will geocode intersections without a zip code being provided.
- https://geocode.localfocus.nl/ is new and promising.
- Peter Aldhous has a great refine geocoder
- GPS Visualizer
- Fusion Tables will do it, but their terms of service say you have to use that data on a Google Map.
- <Geocoder.us> can give you one address at a time or let you pay for batches.
- Carto packages all include some volume of geocoding.
R is super powerful but the learning curve is definitely steep. If you're ready to get to know R, start by getting RStudio. Seriously. Don't try to use R without it. R's ggmap uses Google's geocoding API, which has a limit of 2500 queries per 24 hour period, but that is workable if you have some lead time. You will have to provide a credit card when you sign up for your Google API key so check their policy before you power through thousands of addresses.
Here's a quick and dirty example:
setwd('Path/to/Files')
prek.sites <- read.csv('Pre-K_Sites.csv')
prek.sites$fulladdy <- paste(prek.sites$Address, prek.sites$Boro, ', NY', prek.sites$ZIP)
prek.sites <- cbind(prek.sites, geocode(prek.sites$fulladdy))
write.csv(prek.sites,'PreK_geocoded.csv')
That worked just fine -- it added lat and lon columns to my data frame. NB. paste() is very similar to Excel or Calc's =CONCATENATE()
The Census will take batches of 1000 at a time. Look for the "Address Batch" link in the left-hand nav bar. Have more than 1000? Consider split! Formatting matters. The Census provides a very handy, if cryptic, sample file at http://geocoding.geo.census.gov/geocoder/Addresses.csv:
1,4600 Silver Hill Rd,Suitland,MD,20746
What they mean is that you should submit a CSV in the form id, street address, city, state, zip. If your addresses are broken out differently, take a look at some of my favorite spreadsheet functions for ideas about good tools for dividing and recombining columns.
The Data Science Toolkit includes a geocoder. Mapbox has a tutorial that comes highly recommended.
If you've never installed a Python package before, these CSVkit installation instructions might help. If you've never even used the command line, you might need someone who is game to help you find your way around.
Note: It would be great to add a usage example for dstk!