Search and message streaming

These are two separate tools that arrive together in the same kind of backend. OpenSearch is a search engine: it keeps an inverted index of your text, ranks results, and does the stemming, synonyms and typo tolerance that a SQL LIKE cannot. NATS JetStream is a message streaming server: a service publishes an event, other services consume it later with acknowledgements, retries and replay. One takes read load off the database, the other takes work off the request path.

How we use it

Both appear in Starterflare, and only there. Its search module is backed by OpenSearch and fed from a SearchOutbox table, so an indexing instruction is written in the same database transaction as the change that caused it and picked up afterwards by a worker. That is the transactional outbox pattern, and it exists because the alternative, indexing directly inside the request, silently loses documents whenever the index call fails after the commit. Side effects such as notifications and feed fan-out go over JetStream so a post does not wait for them.

The honest part is that Starterflare never shipped. Postgres, Redis, OpenSearch and NATS is four stateful services, sitting under Kubernetes manifests with autoscaling and ingress, in front of a frontend of eleven pages. We costed that arrangement in what a Postgres, Redis, broker and search stack costs at rest. Everything else we have built chose smaller: ClassProfile does search as case-insensitive substring matching scoped to a tenant, @Heartbeat uses SQL matching over posts, users and hashtags, and Objectify maintains an FTS5 index inside the database. None of them wanted a cluster, and none of them has suffered for it yet.

What it costs you

OpenSearch is a JVM cluster. You size heap, choose a shard count at index creation that you cannot change afterwards, and commit to field mappings you also cannot change afterwards. Discovering that the analyser was wrong for your content means a full reindex, so the rebuild-from-source path has to exist from day one rather than being written in a panic.

The index is a second copy of the truth, and second copies drift. The outbox stops writes being missed; it does not stop a partial reindex, a failed bulk request or a mapping change leaving the two out of step. You need a reconciliation job and a way to compare counts, and both are work nobody budgets for.

Relevance is ongoing rather than a setup task. The first query set looks fine, then real users search for things you did not imagine, and tuning boosts and analysers becomes somebody's recurring job.

JetStream is pleasant to run and easy to get subtly wrong. Acknowledgement modes, redelivery on timeout and the deduplication window all interact, and a consumer that is not idempotent will process a message twice on the day the network hiccups. Stream retention is a storage decision that quietly becomes a cost.

Both bill continuously and both need somebody willing to be paged for them.

When we would choose something else

For a single tenant's worth of documents, the search engine inside the database is usually enough: FTS5 in SQLite and D1, or a tsvector column and a GIN index in PostgreSQL. That covers stemming and ranking without a cluster, and it cannot drift because it is the same database. On a Workers build we use Queues and scheduled work in place of JetStream, since retries and a dead letter queue come with it and there is nothing to run. And when the question is about meaning rather than words, embeddings and vector search answer it better than either.