Aadhib

Case study · In development

Building legal AI that refuses unsupported claims

Making a model decline is harder than making it answer. The retrieval design, the refusal behaviour, and why "no source, no claim" is an architecture rather than a slogan.

Role
Founder · Architecture · Engineering
Published
Reading time
1 min
RAGpgvectorArabic legal corpus
01

Context

Ask any capable general model a Saudi corporate law question and it will produce a confident, well-structured, plausible answer. It will not tell you which provision it came from, and it will not tell you when it is extrapolating. For a lawyer, that output is unusable — not because it is necessarily wrong, but because it cannot be checked.

02

Problem

The default behaviour of a language model is to answer. In legal work the correct behaviour is frequently to decline, or to answer narrowly and cite. Building that means fighting the model's strongest tendency, and doing it reliably rather than by asking politely in a prompt.

03

Constraints

01
Every claim needs a source
Not a bibliography at the end. The specific claim traces to the specific retrieved passage that supports it.
02
The corpus has a boundary
The system has to know where its own knowledge stops, and behave differently on the other side of that line.
03
Arabic legal language
Retrieval quality depends on handling legal Arabic properly — precise, formal and domain-specific.
04
Fluency is not the goal
A less impressive answer that can be verified beats a polished one that cannot.
04

Approach

I made retrieval the gate rather than a hint. The system retrieves from the curated corpus first; generation is constrained to what was retrieved; and where retrieval returns nothing that supports an answer, the system says so instead of filling the gap. The corpus preparation — how legal material is ingested, chunked and indexed — did more for output quality than any choice of model.

05

Architecture

QueryA legal question, in Arabic or English.
Retrieval over curated corpusVector search across prepared Arabic legal material. This is the gate — nothing proceeds without it.
Constrained generationAnswers built from retrieved passages, with sources attached.
Refusal pathWhere retrieval does not support an answer, the system declines rather than generating.
06

Solution

A retrieval-first architecture over a curated Arabic legal corpus, with citations carried through to the output and an explicit refusal path when the corpus does not support a claim. The ingestion pipeline is treated as core product engineering rather than data plumbing, because that is where answer quality is actually determined.

The asymmetry that drives everything

A wrong answer and a refusal are not equally bad. A refusal costs the user a few minutes. A confident, well-written, wrong answer about a filing requirement costs considerably more, and — this is the important part — it does not announce itself. The user finds out later, from someone else.

Once you accept that asymmetry, a lot of design decisions that look overly conservative become obviously correct.

Why prompts are not enough

You can ask a model to cite its sources and to decline when unsure. It will mostly comply, and the failures will be silent and unpredictable. Making the behaviour structural — retrieval gates generation, sources are carried as data rather than as text — turns a tendency into a property.

07

Lessons

  1. 01Refusal is a product feature and an engineering problem. A system that always answers is trivial; one that knows the edge of its own corpus is not.
  2. 02Corpus preparation beats model selection. How the material is ingested, chunked and indexed matters more than which model generates the final text.
  3. 03Citations have to be structural. Attaching sources after generation produces citations that look right and do not support the claim they are attached to.
  4. 04In high-stakes domains, users forgive "I don't know" and do not forgive a confident error. That asymmetry should drive the design.

Stack

What it runs on

Retrieval
Arabic legal corpuspgvectorCorpus ingestion
Generation
Source-grounded answersRefusal behaviour

The project

01MyAdvisor.ai
OPINION"No source, no claim" should be the default for high-stakes AIAI NOTEAI agents are easy to demo. Production agents are different.SYSTEM NOTEThe architecture behind a source-grounded legal AI assistant
All case studies