Model Cards and Documentation for Business AI
Last updated:
The system nobody can explain
Here is a scene we walk into regularly. A business has a model that scores leads, or forecasts demand, or flags suspicious transactions. It was built two years ago by a contractor or an employee who has since left. It still runs. Nobody is quite sure what data it uses, how accurate it is now, or what would happen if it were switched off.
The model may be fine. The problem is that nobody can say so with any confidence, and if a customer, auditor or regulator asks why it made a decision, the honest answer is a shrug.
Model cards were proposed by researchers as a standard way to document machine learning models, and the idea has been widely adopted since. For business AI they solve a practical problem: making sure the knowledge of what a system does outlives the people who built it.
What goes in a model card for a business AI system
Academic model cards can be long. For a business system, these sections cover what people actually need:
| Section | Contents |
|---|---|
| Purpose | What decision or task the model supports, who uses the output, and what it must not be used for |
| Owner | The named business owner and technical owner |
| Model type | In plain terms: a gradient-boosted classifier, a language model with retrieval, a time-series forecast. Plus version. |
| Data | Training data sources, date ranges, known gaps or skews, and any personal data involved |
| Performance | Headline metrics on a held-out test set, with the test set described, and results broken down by important segments |
| Fairness | Groups tested, metrics used, results and actions taken, where the model affects people |
| Limitations | Known failure cases, inputs it handles badly, conditions under which it should not be trusted |
| Human oversight | Where a person reviews output, and thresholds that trigger review |
| Monitoring | What is tracked in production and the alert thresholds |
| Change log | Date, version and reason for each retrain or significant change |
For systems built on a third-party language model, the card should also name the provider and model family, the prompts or configuration in use, and the evaluation set used to check behaviour when the provider updates the model.
Limitations: the section that matters most
Most model cards we review have a thorough performance section and a limitations section of two vague sentences. That is backwards. The performance numbers tell you how the model does on average. The limitations tell the people using it when not to trust it, which is what prevents harm.
Good limitation statements are specific enough to act on:
- 'Accuracy drops sharply for invoices from suppliers not seen in training; route new suppliers to manual review for the first three months.'
- 'Forecasts for products with fewer than 12 months of sales history are unreliable and should not drive purchase orders.'
- 'Scores for customers in Northern Ireland are based on very few examples and should be treated as indicative.'
- 'The model has not been tested on handwritten documents.'
A limitation you have written down is a risk you are managing. One you have not is a surprise you have scheduled.
A model card is not enough on its own
The model card describes the model. Keeping a business AI system running needs a second, more operational document, often called a runbook. It answers the questions someone asks at 4pm on a Friday when something looks wrong:
- Where does the system run, and how is it deployed?
- What does normal look like in the monitoring dashboard?
- What are the common alerts and what should be done for each?
- How is the model retrained, by whom, and how is the new version tested before release?
- How is it rolled back?
- What happens to the business process if the system is switched off? Is there a manual fallback?
Pair both documents with the system's logs. Decisions that can be traced back to inputs and model versions are far easier to explain; our post on audit trails for AI integrations covers what to record.
How much documentation is enough?
Proportion it to consequence. Our rough guide:
| System type | Documentation |
|---|---|
| Internal productivity aid, human reviews every output | One-page card: purpose, owner, data, limitations |
| Automated process, errors caught downstream | Full model card plus runbook |
| Customer-facing or affecting decisions about people | Full card with fairness section, runbook, evaluation records and change approvals |
| High-risk under the EU AI Act | All of the above as a foundation, plus whatever the provider or deployer obligations require; take legal advice |
Over-documentation is a real failure too. A forty-page document for a ticket-routing model will not be read and will not be updated, which makes it worse than a short one that is.
Keeping it true after launch
Documentation rots faster than code. Three habits keep model cards honest:
- Make updating the card part of releasing a new version. No retrain goes live without a change log entry and refreshed performance figures.
- Generate the numbers automatically. Performance tables should come from the evaluation pipeline, not be typed by hand. This fits naturally into a sensible MLOps setup for mid-sized companies.
- Review the card at the same cadence as the model. If the model is reviewed quarterly, so is its documentation, including whether the limitations still hold.
Store it with the code, in version control, so the card that describes version 4 is the one sitting next to version 4.
How we handle it
Every model SpiderHunts delivers through our machine learning service ships with a model card and runbook written during the project rather than at the end. Writing them as we go has a useful side effect: when the limitations section is hard to write, it usually means the testing was not thorough enough yet. Where a model affects people, the card includes the results of bias testing, including what we found and changed.
If you are buying rather than building, ask the vendor for their equivalent. A supplier that cannot tell you what their model is for, what it was tested on and where it fails is asking you to take a lot on trust.
Frequently asked questions
What is a model card?
Who should write the model card?
Do we need model cards for AI tools we buy?
How long should a model card be?
Does the EU AI Act require model cards?
Inherited an AI system nobody documented?
We can review how it works, test it on current data and write the model card and runbook it should have come with.