The architecture behind a source-grounded legal AI assistant

Building MyAdvisor.ai forced a set of architectural decisions that I think generalise to any high-stakes retrieval system. This is the shape of it, without the parts that should stay private.

Retrieval is a gate, not a hint#

The common RAG pattern retrieves some context, puts it in the prompt, and asks the model to use it. The model usually does. It also freely supplements from its own parameters, and you cannot tell from the output which sentence came from where.

For legal work that is not acceptable, so retrieval has to gate generation rather than inform it. If retrieval returns nothing that supports an answer, the system does not generate one. That is an architectural property, not a prompt instruction, because prompt instructions fail silently and unpredictably.

Corpus preparation beats model selection#

This is the least exciting and most important thing I can say about RAG.

Teams spend a lot of energy choosing a model and comparatively little on how material enters the corpus. That is backwards. How documents are segmented, what metadata travels with each segment, how hierarchical structure is preserved, how cross-references are handled — those decide whether retrieval surfaces the right passage. If it does not, no model rescues the answer.

Legal Arabic makes this sharper. It is precise, formal and structurally specific. Chunking it as though it were prose loses the structure that makes a provision meaningful.

Citations have to be structural#

There are two ways to attach sources to an answer. You can ask the model to cite, or you can carry the source identity as data through retrieval and attach it to the claim mechanically.

The first produces citations that look right and are sometimes attached to the wrong claim, which is worse than no citation — it manufactures unearned confidence. The second is more work and it is the only version that holds up when a professional actually checks.

The refusal path is part of the design#

Most systems treat "no answer" as an error state. Here it is a legitimate outcome with its own path: the system says the corpus does not support an answer, and ideally says what it does have.

Building that requires deciding what "insufficient support" means concretely — a threshold, a retrieval quality signal, something checkable. It is a design decision, and dodging it means quietly defaulting to always answering.

The stack, briefly#

Next.js and NestJS across a Turborepo, with Expo for mobile. PostgreSQL with pgvector for retrieval, Redis for caching, MinIO for document storage.

None of that is exotic and none of it is the interesting part. Vector search in Postgres rather than a dedicated vector database is a deliberate simplification — one datastore, one backup story, one operational surface. The interesting part is everything described above, which would be the same regardless of which of these components you swapped out.

The one rule#

No source, no claim.

It reads like a slogan. It is actually a constraint that decides the retrieval design, the storage model, the output format and the failure behaviour. Every one of those is different in a system that is permitted to guess.