Connecting residents with their communities through location-based search platforms

Much of our lives, and the lives of people we create services for, revolve around the unique context of our places. Our communities and neighbourhoods, perhaps now more than we ever realised before, are more than where we live but also where people work, socialise and learn. A common need we find when working with local authorities is a need to connect residents with services, organisations or events beyond the council and in their local area. We see user stories like:

• I need to find services in my local area
• I need to know what’s going on around me
• I need to see the closest events to me

Increasingly, we expect the apps and platforms we use to be location-aware, highlighting services within a specific geographic area. Be that regionally, city-wide or at a neighbourhood level. Driven by apps like Airbnb, Uber or Google Maps, this raised expectation is a dataset we can build into council platforms, supporting residents with the kind of connectivity they expect.

In this post, we’ll look at a few ways to add location-aware search, comparing two main approaches: one with Ruby on Rails with PostgreSQL, and one with Node.js and MongoDB.

First, let’s consider the kind of data we’ll need.

Preparing our data

Let’s say that our users are parents, and we want to show them nearby childminders. We might have data like this:

• childminder name
• contact phone number
• street address
• town
• postcode

The first step is geocoding the data: turning those address fields into coordinates we can use.

Google has a geocoding API we’ll use. Because it has the power of Google Maps behind it, it’s fairly resilient to slight misspellings and misformatting that will probably be common to any real-world dataset.

There are plenty of alternatives, but they all have idiosyncrasies. It’s worth playing around and seeing which one gives the most valuable results for your data.

Some of these APIs can get expensive if we have thousands of data points to geocode, but it’s possible to use Google Sheets to do the hard work for us.

Once we have a fully geocoded dataset, it’s time to think about querying it.

The Rails/Postgres approach

Let’s look at how to work with our data in a fresh Rails app.

We’ll start by making a new model called Service, which we’ll add extra columns onto to represent our coordinates:

• rails g model Service name:string phone:string address:string town:string postcode:string latitude:float longitude:float

There’s a great Rails gem called geocoder which is easy to set up and adds all sorts of useful geospatial methods to our models.

Getting the straight-line distance to each result is normally enough to meet the user’s need, and it substantially simplifies the computations we need to make.

We can plug our latitude/longitude pairs into the haversine formula, which accounts for the curvature of the earth (which becomes an important source of error on distances greater than about 20 kilometres).

The alternative would be to call an external directions API, like Google’s, and calculate real-world distances, but that’s not practical for large datasets if there’s no way of knowing where the user wants to search from.

Luckily Rails abstracts all that into the background, so once we’ve got our prepared data imported, getting results by distance from a location is simple:

• Service.near("SE1 9RG")

That method will both geocode your search query into a latitude/longitude and do the maths to sort the results.

For simple use cases, this will be enough, but on a recent project we bumped up against some limitations that had us looking for a more customisable approach.

Let’s extend our childminders example from earlier to consider larger organisations that might offer the same service from lots of locations.

It’s still the service that we’re interested in, but now the coordinates we need to search on are in a separate database table, and there can be many coordinate pairs per service. The geocoder gem doesn’t support this use case out of the box, and it’s quite a slow, complex database query if we code it manually.

The Node.js/MongoDB approach

MongoDB is a document store database — rather than traditional rows and columns, it stores blobs of JSON in a format that can be easily manipulated in Node.js.

Mongo also has excellent support for geospatial queries, with operators like $nearSphere.

Because of the document model it follows, it’s also great at querying nested data, like our one-service-to-many-locations structure.

1. Start using geoJSON

Although we can also use simple coordinate pairs, MongoDB encourages geoJSON for formatting geospatial data. Here’s a simple geoJSON object which we could nest inside our data for querying:

• {
  type: "Point",
  coordinates: [-0.0909866, 51.5046914]
  }

GeoJSON coordinates are always ordered longitude-first, which sometimes trips newbies up.

2. Create an index

The other extra step needed to activate geospatial querying is to create the right index. This can be done programmatically, or with a GUI app like MongoDB Compass.

Making an index using the Node.js driver could look like this:

• db.collection("services").createIndex({
  "locations.geometry": "2dsphere"
  })

We’re using dot notation to query nested data, in this case, an array of locations on each service document, where each one has a geoJSON object stored on the geometry key.

3. Query the data

Once we’ve got our data prepared and indexed, we can query it using the MongoDB Node.js driver:

• db.collection("services").find({
  "locations.geometry": {
  $nearSphere: {
  $geometry: {
  type: "Point",
  coordinates: [-0.0909866, 51.5046914]
  }
  }
  }
  })

In this example, we’re using dot notation, geoJSON and the $nearSphere operator together.

This is much more verbose than the Rails example and doesn’t automatically handle geocoding the user’s location, but it’s a lot more flexible, and (if we plan it the right way) potentially speedier too.

Improving it further

GeoJSON is a powerful format that’s not limited to representing points. It can represent complicated polygons and shapes, like the coverage area of a service.

This opens the door to things like showing a user all the organisations that serve their address. This is an increasingly compelling use case in a world where fewer people are leaving home and physical locations are shut.

If you need to combine geospatial queries with intelligent keyword searching, you might want to consider Elasticsearch or Algolia instead of the solutions covered here.

Lastly, this kind of location-based search often works well paired with geolocation: fetching the user’s latitude and longitude based on their device’s sensors — wireless connections and GPS — rather than making them enter it manually.