Quality is not uniform
A model at 94% on English documents might be 91% on German, 88% on Polish and materially worse on languages with less training data.
A single global accuracy figure hides that, which is why we always evaluate per language and report per language.
What to do about the variation
- A separate evaluation set per language, built by a native speaker
- Confidence thresholds set per language, not globally
- More human review for the weaker languages, deliberately
- Honest reporting per language rather than a blended average
Setting one threshold across five languages means the strong ones are over-reviewed and the weak ones under-reviewed. Per-language thresholds fix both at once.
Detect, do not assume
Language detection before processing, with a fallback to human routing when detection is uncertain. Mixed-language documents are common and they confuse pipelines that assume one language per item.
Detection is cheap and reliable; assuming based on the customer's country is neither.
Translation is a separate decision
- Process natively where quality allows — usually better
- Translate then process where the model is weak in that language
- Translate the output for a reader who needs a different language
Each adds a step where errors compound, so use as few as the quality bar allows.
Do not draft in a language nobody reads
If no one on your team can check Hungarian, do not have the system draft Hungarian replies for review. Route those to a translator or a partner.
Review that cannot actually review is worse than no review, because it produces a record of approval nobody performed.