The Architecture of an AI Application, Explained Simply

Five layers, one of which is the model

Clients often imagine an AI app as a model with a website in front of it. In practice the model is one component among several, and the others determine whether it works.

Understanding the layers makes it much easier to tell where a quote is thin.

1. Intake

Getting the input in: a mailbox watcher, an upload, an API, a form, a scanner. This layer normalises whatever arrives into a consistent internal shape.

It is unglamorous and it is where a surprising share of the reliability comes from. A document that arrives as an email attachment inside a forwarded message needs handling before anything intelligent happens.

2. Retrieval

Finding the material the model needs to answer well: your documents, the customer's history, the relevant policy. Splitting content into chunks, indexing it by meaning, and pulling the best few at question time.

Most quality problems in AI applications are retrieval problems, not model problems. If the right context is not found, the best model in the world answers confidently from the wrong material.

3. The model call

The prompt, the retrieved context, and the request. This layer also decides which model to use — a cheap fast one for classification, a stronger one for reasoning — and enforces a structured output format so the result can be checked programmatically.

4. Validation and routing

• Does the output parse into the expected structure?
• Does it agree with what we already know — does the total match the lines, does the supplier exist?
• Is confidence above the threshold for this field?
• Route: proceed automatically, send to review, or reject

This layer is rules, deliberately. It needs to be readable, changeable and explainable.

5. The human interface

Where a person sees what needs attention, corrects it in seconds, and where the correction is captured so the same case is never queried twice.

Systems without this get abandoned. Being unable to fix an obvious error is the single most infuriating property software can have.

Frequently asked questions