One request, three safeguards: an AI architecture teardown

This is a teardown of a small architecture we built as in-browser demos on our Our Work page. It shows three safeguards that sit between a person's question and an AI model, and the decisions behind each one. The demos use made-up data and simple rules so that anyone can try them. A production system would use stronger methods, and we say where.

The request, end to end

• Mask: sensitive values such as emails, phone numbers and ID numbers are swapped for placeholders like [EMAIL_1] before anything else sees the text.
• Assemble: a fixed budget decides which tools, earlier messages and documents fit, and leaves the rest out.
• Route: easy questions go to a fast, cheap model and hard ones to a slower reasoning model.
• Restore: placeholders in the reply are swapped back for the real values, and every step is logged.

Decision 1: mask personal data before anything leaves your system

Context. A model call sends text to a service outside your own systems, and question text often contains personal details that the model does not need in order to help.

Decision. Replace each sensitive value with a stable placeholder, keep the originals in a vault inside your own boundary, and put them back in the reply. The same value always gets the same placeholder, so the model can still tell who is who.

Consequences. The model cannot leak what it never saw, and logs hold placeholders instead of real values. The cost is that pattern matching only finds values with a recognisable format. Names and addresses need a list or a trained model, and the vault itself becomes a sensitive store that needs encryption, access control and a retention rule.

Decision 2: give every request a context budget

Context. A prompt has a size limit, and more text is not always better. A bigger prompt is slower and costs more, and unrelated material can distract the model.

Decision. Always include the instructions and the question, keep a slice free for the answer, score every tool, earlier message and document against the question, and pack the best ones until the budget is full.

Consequences. Size and cost become predictable, and you can show exactly what was left out and why. The quality depends on the scorer: the demo matches words, while a real system would use embeddings and a re-ranker. Dropping old messages also loses detail unless they are first summarised.

Decision 3: send easy questions down a fast lane

Context. Most questions are simple, and a reasoning model is slower and more expensive than they need.

Decision. Read each question for clues of difficulty, such as length, requests for reasoning, errors or high-stakes topics, and send it to the fast or the reasoning model. One setting decides how many clues are enough to escalate.

Consequences. Average cost and waiting time fall, but some hard questions slip into the fast lane. The usual remedies are to escalate when the fast model is unsure or when a check on its answer fails, and to review a sample of what was routed fast.

What the demo numbers do and do not show

In the Triage Router demo, with its 20 invented sample questions and illustrative costs of 1 unit for the fast model and 20 for the reasoning model, a setting of 2 points gives about 48% lower cost with no risky calls. A setting of 3 points gives about 76% lower cost but lets 4 hard questions into the fast lane.

Why the order matters

Masking comes first so that every later step, including logs, caches and the model, only ever sees placeholders. Assembly comes before routing so that whichever lane is chosen receives a prompt of known size. Restoring comes last, at the edge, so real values exist in as few places as possible.

Try the pieces yourself

• PII Unmasking Proxy: paste text and watch personal values become placeholders and come back
• Context Budget Assembler: shrink the budget and see what gets left out
• Triage Router: move the dial and watch cost and risk change
• Demo manual: step-by-step instructions for all three

If you would like a design like this for your own product, with the right methods for your data and your traffic, we would be glad to talk it through.