Running AI Features in More Than One Language

Quality is not uniform

A model at 94% on English documents might be 91% on German, 88% on Polish and materially worse on languages with less training data.

A single global accuracy figure hides that, which is why we always evaluate per language and report per language.

What to do about the variation

• A separate evaluation set per language, built by a native speaker
• Confidence thresholds set per language, not globally
• More human review for the weaker languages, deliberately
• Honest reporting per language rather than a blended average

Setting one threshold across five languages means the strong ones are over-reviewed and the weak ones under-reviewed. Per-language thresholds fix both at once.

Detect, do not assume

Language detection before processing, with a fallback to human routing when detection is uncertain. Mixed-language documents are common and they confuse pipelines that assume one language per item.

Detection is cheap and reliable; assuming based on the customer's country is neither.

Translation is a separate decision

1. Process natively where quality allows — usually better
2. Translate then process where the model is weak in that language
3. Translate the output for a reader who needs a different language

Each adds a step where errors compound, so use as few as the quality bar allows.

Do not draft in a language nobody reads

If no one on your team can check Hungarian, do not have the system draft Hungarian replies for review. Route those to a translator or a partner.

Review that cannot actually review is worse than no review, because it produces a record of approval nobody performed.

Frequently asked questions