Scoping a First Version of an AI Application

The risky assumption is different

In a conventional MVP the question is whether anyone wants it. In an AI MVP there is usually a second, sharper question: will it be accurate enough on our actual data to be useful?

That question can be answered far more cheaply than a product, and it should be answered first.

Answer accuracy before building anything

1. Take fifty real cases with known correct answers
2. Run the intended approach against them
3. Measure per field or per category, not overall
4. Show the results to whoever will use it and ask whether that is good enough

This costs days rather than weeks. If accuracy on fifty real cases is poor, no amount of engineering rescues the use case — the problem is the problem, not the implementation.

What the MVP must include

• The core workflow end to end, actually working
• An evaluation set that runs on every change
• A review interface, because the first version will need humans
• Instrumentation showing completion and correction rates
• A way for users to report that something is wrong

What can wait

1. An admin panel — a developer running a query is adequate for the first fifty users
2. Multiple roles — permissions multiply testing faster than anything else
3. Self-service billing — invoice the first customers manually; it is better research
4. Additional document types or languages — prove one first
5. Automatic actions — start with suggestions a human approves

Manual behind the curtain is legitimate

If the hard part is a process, run it by hand at first. The customer sees a form and receives a result; behind it a person does the work with AI assistance.

It is the cheapest possible test of whether the result is worth paying for, and it produces exactly the labelled data you need to automate later.

Frequently asked questions