I struggle with FTS inside SQL (SQLite and MSSQL). There is often a fairly significant impedance mismatch between the relational concerns and how the documents need to be stored.

I've always preferred to use SQL as the system of record and then build/maintain an external Lucene index. Do we think these integral FTS capabilities are at the point where a hybrid architecture doesn't make sense anymore? How much customization exists in this provider?

▲

zombodb 33 minutes ago | parent | next [-]

I’m one of TIN’s developers and if you google my username you’ll see I’ve been in this space for a long time.

The answer to your first question is simply: yes

As far as your second question, what customization do you need that you believe TIN or PlanetScale doesn’t provide? These are things we can do, with alacrity.

▲

gfody 31 minutes ago | parent | prev [-]

in my experience it’s pretty common to find big inverted indexes for text directly in the database - not necessarily large docs but certainly free text records in volume. using bm25 and unicode’s breakiterator is a very good way to build it. like putting lucene in the database basically - makes a lot of sense when the database is already large. places that bend over backwards to move search out of the db are usually trying to avoid having a very large db (and often end up with one anyway, getting the worst of both worlds)

	▲	zombodb 22 minutes ago \| parent [-]
		They also end up with all the infrastructure and processes necessary to keep the external search system in sync, resync/reindex, pkey shipping back to their source of truth in queries, application-side joins and enrichment between both sources. It’s brutal. Having everything in one place eliminates entire classes of development and especially operational problems.