When Postgres full-text search isn't enough: integrating Elasticsearch/OpenSearch with Django — modelling documents, keeping the index in sync, tuning relevance with boosts and fuzziness, facets, and zero-downtime reindexing with aliases.
PostgreSQL full-text search is excellent until it isn't — once you need typo tolerance, relevance tuning, faceted navigation, and sub-100ms search across millions of documents, a dedicated search engine earns its keep. This tutorial covers integrating Elasticsearch or OpenSearch with Django: when to graduate from Postgres, how to model and index documents, how to keep the index in sync with your database, and how to tune relevance and run zero-downtime reindexes.
Postgres full-text search is the right first choice: no extra infrastructure, transactional consistency, and genuinely good for straightforward keyword search. You outgrow it when you need fuzzy matching and typo tolerance, sophisticated relevance scoring you can tune, faceted search with fast aggregations, autocomplete-as-you-type, or search latency that stays flat as data grows into the millions. Elasticsearch and OpenSearch (an open-source fork of Elasticsearch, API-compatible for most purposes) are purpose-built for exactly this. The tradeoff is real: a second datastore to run, secure, and keep in sync. Do not add it before Postgres actually hurts.
The engine holds an index of documents — denormalized JSON representations of the things you search. This is the key mental shift from a relational database: you do not join at query time, you flatten the data you need into each document at index time. Your Postgres database stays the system of record; the search index is a derived, eventually-consistent projection of it that you can rebuild at any time.
The most ergonomic option is django-elasticsearch-dsl, which maps Django models to search documents declaratively. Under it sits the lower-level elasticsearch-py client, which you can always drop to for complex queries.
# documents.py
from django_elasticsearch_dsl import Document, fields
from django_elasticsearch_dsl.registries import registry
from .models import Article
@registry.register_document
class ArticleDocument(Document):
author_name = fields.TextField(attr="author.get_full_name")
tags = fields.KeywordField(multi=True)
class Index:
name = "articles"
settings = {"number_of_shards": 1, "number_of_replicas": 1}
class Django:
model = Article
fields = ["title", "body", "published_at"]
The distinction between TextField and KeywordField is fundamental: text fields are analyzed (tokenized, lowercased, stemmed) for full-text matching, while keyword fields are stored verbatim for exact filters, sorting, and aggregations. Use text for the body you search, keyword for tags, categories, and status you filter and facet on.
An index is only useful if it reflects current data. django-elasticsearch-dsl wires Django signals so saving or deleting a model updates the index automatically. That is convenient but has two traps: signals fire inside your request, adding latency and a failure mode if the cluster is down, and bulk operations that bypass save() (like queryset.update()) do not fire signals at all.
# Initial / full rebuild
python manage.py search_index --rebuild
# In production, prefer async indexing: signals enqueue a Celery task
# that does the actual ES write, so a slow or down cluster never
# blocks or breaks the user's request.
The robust pattern is to make the signal enqueue a background job rather than write synchronously, and to run a periodic full reindex as a safety net that heals any drift from missed updates. Treat the index as eventually consistent and design the UI to tolerate a few seconds of lag.
Search queries combine a scoring part (how well does this match?) with a filtering part (is this even eligible?). The bool query expresses this: must clauses score, filter clauses constrain without affecting the score and are cacheable.
from elasticsearch_dsl import Q
search = ArticleDocument.search()
search = search.query(
Q("bool",
must=[Q("multi_match", query=term, fields=["title^3", "body"], fuzziness="AUTO")],
filter=[Q("term", status="published"), Q("range", published_at={"gte": "now-1y"})])
)
results = search[:20].execute()
Two relevance levers appear here. The title^3 boost says a match in the title counts three times as much as one in the body. And fuzziness="AUTO" gives typo tolerance, matching "djnago" to "django". Putting status and date in filter rather than must both keeps them out of scoring and lets Elasticsearch cache them.
Relevance is where a search engine pays off, and it is iterative. Analyzers decide how text is broken into tokens — a good analyzer lowercases, removes stop words, and applies stemming so "running" matches "run". Field boosts weight important fields. Function scoring lets you blend text relevance with business signals like recency or popularity, so a fresh, widely-read article outranks an old one on an equal text match. The workflow is to search with real queries, look at what ranks and what should have, and adjust — relevance is tuned against your data and users, never guessed once.
The faceted counts you see on search results ("Django (42), Python (31)") are aggregations, computed in the same round trip as the search. Because they run on keyword fields, this is fast, and returning results plus facet counts together is what makes filtered search feel instant.
Sooner or later you must change a mapping — add a field, change an analyzer — and mappings are largely immutable, so you rebuild. Do it without downtime using an alias. Your app always queries the alias articles; behind it sits a concrete index like articles_v3. To migrate, build articles_v4, backfill it fully, then atomically repoint the alias. Searches never see a half-built index, and rollback is just repointing the alias back.
Instant suggestions as the user types are a search-engine speciality that Postgres cannot match cheaply. The technique is edge n-grams — indexing "django" also as "d", "dj", "dja", "djan" — so a prefix matches immediately. Elasticsearch also ships a dedicated completion suggester optimised for this, backed by an in-memory structure for microsecond lookups. Keep the autocomplete index separate and lean; it answers a different question ("what might they mean?") than full search ("what matches?").
Showing users why a result matched — the query terms bolded in a snippet — measurably improves perceived relevance. Elasticsearch returns highlighted fragments for you.
search = ArticleDocument.search().query("match", body=term)
search = search.highlight("body", fragment_size=150, number_of_fragments=1)
for hit in search.execute():
print(hit.meta.highlight.body) # ...text with <em>term</em> wrapped...
Render those fragments instead of the raw field and the result list immediately feels more relevant, even when the underlying ranking is unchanged.
The naive from/size pagination breaks down at depth: asking for page 500 forces every shard to collect and sort 500 pages of results, which is slow and memory-hungry, and most clusters cap it at 10,000 results outright. For deep or infinite scrolling, use search_after, which pages by the sort values of the last hit rather than an offset. It is stateless, cheap at any depth, and the correct default for feeds. Reserve from/size for shallow, page-numbered UIs.
They share ancestry and most of an API, so your Django integration looks nearly identical either way, but the difference that matters is licensing. Elasticsearch moved to a non-open-source licence, prompting the OpenSearch fork, which stays Apache-2.0 and is the default on AWS. Choose on licensing posture, managed-service availability, and which client libraries you trust — not on query syntax, which barely differs for typical use. Whichever you pick, pin the client library to the matching major version, because cross-version clients fail in confusing ways.
A search cluster is infrastructure with its own failure modes: heap pressure, shard rebalancing, and the dreaded split brain. Watch JVM heap usage, per-node CPU, and query latency percentiles, and alert before the cluster turns yellow. Size shards to a few tens of gigabytes each — many tiny shards waste overhead, while huge shards slow recovery. Treating the cluster as something you monitor and capacity-plan, rather than a box you install once, is the difference between search that stays fast and search that mysteriously degrades at month six.
Relevance is language-specific. An English analyzer stems "running" to "run"; a Dutch or German one handles compound words and different stop words entirely. If your content spans languages, analyze each field with the right language analyzer rather than a single default, and consider per-language fields for truly multilingual corpora. Getting analyzers right is often a bigger relevance win than any query tweak, because it changes what "matches" at the token level.
Two security concerns travel with a search engine. First, the cluster itself: never expose an unauthenticated node to the network — enable authentication and TLS, and keep it on a private subnet, because an open search node is a well-known breach vector. Second, the query path: never build queries by string-concatenating user input into the query DSL. Use the client's structured query objects so user terms are always data, never structure — the search-engine equivalent of parameterized SQL, and the defense against query injection.
Pure text relevance is rarely what users want at the top — a perfect keyword match on a three-year-old, never-read article should usually lose to a strong match on fresh, popular content. Function scoring lets you multiply text relevance by business signals: a recency decay so newer documents rank higher, a popularity factor from view counts, or a hand-set boost for featured items. The craft is balance — weight the signals too hard and you bury genuinely relevant results under whatever is merely popular. Tune it against real queries, watch what moves to the top, and adjust the weights until the ranking matches what a knowledgeable human would pick.
Users and content rarely use the same words. Someone searches "js" but your articles say "JavaScript"; they type "k8s" and you wrote "Kubernetes". A synonym filter in the analyzer expands these at index or query time so the match happens anyway, and a curated synonym list is one of the highest-value, lowest-effort relevance improvements you can ship. Maintain it as data you can update without reindexing where possible, and grow it from real query logs — the searches that returned nothing are a direct list of the synonyms and misspellings your users expect you to understand.
Watch for sync lag presented as a bug — a just-created record not appearing in search — which you handle in the UI, not by making indexing synchronous. Beware mapping explosions from indexing arbitrary user JSON with dynamic mapping, which creates thousands of fields and degrades the cluster; disable dynamic mapping on untrusted data. Secure the cluster: OpenSearch and Elasticsearch bind to loops of trust, and an exposed, unauthenticated node is a classic data breach. And size shards deliberately — over-sharding a small index wastes resources as surely as under-sharding a huge one strains them.