It took about four months. Correctness was reviewed continuously rather than only after implementation. We built the system incrementally in small slices, with humans handling the architecture, and review while AI performed much of the implementation and front end is fully doe by humans.

The ingestion side includes durable asynchronous processing, recovery and replay, idempotency, strict ordering, structure aware segmentation, parent child relationships, and batched local embedding. The retrieval side includes hybrid (rrf), text and vector search, document versioning, collection and category scoping, document level ACL enforcement, parent expansion, provenance, and citation tracking.

Your estimate is fair. But, reliable WAL and retrieval layer are each substantial projects. We did not build the underlying database query engine from scratch, but the correctness work around ingestion, recovery, hierarchy, security, ranking, and provenance was still significant.

The project now has more than 3,000 automated tests, including coverage for recovery, concurrency, ordering, ACLs, versioning, and retrieval behaviour. Tests are not proof of correctness, but incremental implementation, explicit invariants, failure testing, and continuous human review are how we built confidence in it. We also had script to automate live endpoint tests with real data samples, this help largely to mitigate issues after a refactoring or enhancement. total production code is 80k LOC and rest is test coverages. One thing to say after seeing the results are, many of the RAG framework does it wrong. We got exceptional results and we also have agentic retrieval for answering complex queries. We designed it in our own way because government systems cannot afford false results, especially when the product is intended for crime investigation or compliance management in banks.

Edit: I forgot to mention that we also support PostgreSQL.

After building this system, we concluded that vector dimensionality is only a small part of retrieval accuracy. We use 384 dim embeddings and still achieve excellent results.

The tests are based on actual expected outputs, not generated only to increase coverage. For every feature, we have a phase in TASK Ledger with scope, objective, goals and outcome. And the final part will be test coverage with expectations. I cannot claim that a human has reviewed every test line by line. However, humans defined and reviewed the expected behaviour, architecture, invariants, and feature outcomes against which those tests were written. Also, when a bug is reported, we add a regression and prove the bug and then fix and test should pass.

We also use a headless API with a fully decoupled frontend, and the frontend is written entirely by developers. This helps catch additional issues during integration because every endpoint is exercised and validated through the actual userfacing workflows.

I may not be able to convince developers about the process we follow, but we got it working, and tested for concurrency, load and fanout behaviour. This is our primary requirement, as this is targeted towards large scale government customers. Regarding future tests, they will be added for new features, reported bugs, newly discovered failure modes, and regressions. This level of test coverage also helps us use AI without allowing significant drift in the codebase.