Implementing a simple, effective search in Firebase with just Firestore

Photo by Clem Onojeghuo on Unsplash

When I was building my first full-scale Firebase app, I quickly discovered that search is one of the drawbacks of firestore. From the docs:

Cloud Firestore doesn’t support native indexing or search for text fields in documents.

In other words, if you are coming form MySQL, there is no “LIKE” query. Or if you are coming from MongoDB, there is no RegExp queries. We have more constraints.

The docs go on to suggest using a third-party paid service called Algolia. This is disappointing, since you probably don’t want to pay for a separate service.

However, here’s one way to setup a pretty effective search with just firestore. In this context I’m using Angularfire, but it will probably be useful for any firestore client. Let’s build it!

A unicode trick

Range queries get us part of the way. In combination with a special unicode character, \uf8ff we can string-match titles with a definite start, but uncertain end:

this.afs.collection<Book>(‘books’, ref =>
  ref.orderBy(‘title’).startAt(term).endAt(term + ‘\uf8ff’)
).valueChanges();

The docs explain:

The \uf8ff character used in the query above is a very high code point in the Unicode range. Because it is after most regular characters in Unicode, the query matches all values that start with a term.

But wait! If the user types “the black pearl” it won’t find “The Black Pearl” because the case doesn’t match.

Not only that, but if they type “black pearl” it won’t match without the “The” in front. A solution follows…

Special document properties just for search

Let’s add 2 extra title properties to our document to make search more flexible, and make sure they are included when it gets added to a collection: title_lower and title_partial.

const bookDoc: Book = {
  title: book.title,
  title_lower: book.title.toLowerCase(),
  title_partial: this.partialTitle(book.title),
  ...
};

One stores the title lowercased, and the other stores a partial title. The partial title method just removes the first word if the title has more than one word:

partialTitle(title: string): string {
  const parts = title.split(' ');
  if (parts.length > 1) {
    parts.shift();
    return parts.join(' ').toLowerCase();
  } else {
    return title.toLowerCase();
  }
}

The Query

Finally let’s combine multiple queries into one, to search 2 title properties at once. Our search method will also find docs typed with uppercase characters in search terms, because we just lowercase everything:

search(term: string): Observable<Book[]> {
  term = term.toLowerCase();
  // search both title_lower and title_partial
  const title$ = this.afs.collection<Book>('books', ref =>
    ref.orderBy('title_lower').startAt(term).endAt(term + '\uf8ff')
  ).valueChanges();
  const partial$ = this.afs.collection<Book>('books', ref =>
    ref.orderBy('title_partial').startAt(term).endAt(term + '\uf8ff')
  ).valueChanges();
  return merge(title$, partial$).pipe(
    pairwise(),
    map(([title, partial]) => {
      const result = [...title, ...partial];
      // dedupe
      return result.filter((thing, index, self) =>
        index === self.findIndex((t) => t.title === thing.title)
      );
    })
  );
}

And there you have it. We’re using rxjs’ merge to combine our 2 queries, and sending their results through the stream in pairs, so we can concatenate the results and then dedupe them.

This is ideal for a typeahead because the user will likely find what they are looking for before they type the entire title. Is it perfect? No. For instance, if the user starts typing a word in the title that is not the first or second word, they won’t find it. Does it save you from paying for a third party service? Yes.

Make it even more powerful?

To really juice this up, we could combine a 3rd stream. Keep in mind billing — a collection query result charges a read for each doc returned in the result. We are already using 2 queries per search. You may want to include a search button instead of using a typeahead to reduce queries.

This query uses the array-contains matcher. You might think this could replace everything else, but it can’t. The reason is because the entire term has to match. The would be too restrictive alone, but could find a word match anywhere in the title.

const word$ = this.afs.collection<Book>(‘books’, ref =>
  ref.where(‘title’, ‘array-contains’, term)
).valueChanges();

You may choose to include this, but if not you still have a powerful search that’s cost effective.

If you’d like a demo of this, go to Podfan and try searching a podcast.