Building Multilingual AI That Actually Works in Arabic and English

The most common architecture I see: English embeddings, English prompts, and a translation layer at the boundary. It looks bilingual. It performs terribly.

• Translation loses idiom and context; retrieval matches the wrong docs.
• Prompts written in English don't leverage the model's Arabic reasoning capacity.
• The translation step adds latency and cost on every call.
• Users who code-switch ("ابعت لي report عن Q3") get bounced.

The right architecture treats Arabic as a first-class language throughout the pipeline.

Benchmark on your data, not just MIRACL or MTEB. See vector databases compared 2026 for the storage layer.

The temptation is to run every Arabic string through the same normalizer. That's often wrong. Some normalizations preserve meaning; others destroy it.

import re, unicodedata

def normalize_arabic(text: str) -> str:
    text = unicodedata.normalize("NFC", text)
    text = re.sub(r"[أإآ]", "ا", text)
    text = re.sub(r"[ً-ْ]", "", text)  # tashkeel
    text = text.replace("ـ", "")                  # tatweel
    text = re.sub(r"s+", " ", text).strip()
    return text

Users write in dialect. Documentation is in MSA. Retrieval fails silently.

Real example: a user asks عايز اعمل ايه لو الفاتورة ما وصلتش؟ (Egyptian). Your KB has a section titled الإجراءات في حالة عدم استلام الفاتورة (MSA). Cosine similarity: mediocre.

Two production patterns work:

1. Query rewriting: run the user query through a small model that produces both the original and an MSA rewrite; retrieve on both, union results.
2. Dialect-tagged index: enrich each chunk with dialect-equivalent phrasings at ingestion. Costs more storage; huge recall win.

Details in RAG that actually works — same principles, applied to language variance.

Arab users routinely code-switch and write Arabic in Latin script (Arabizi: eb3at le report). Your system should handle both without users learning to speak 'properly' for the model.

• Detect script mix at query time; route through both an Arabic and an English embedding if confidence is split.
• Maintain a small Arabizi → Arabic phrase dictionary for common domain terms; expand queries before embedding.
• In prompts, tell the model explicitly: 'Users may mix Arabic and English. Understand both. Answer in the primary language of the user's question.'

Structured system prompts in the target language outperform English prompts by 5–15% on Arabic tasks, in every eval I've run. Write two versions of your system prompt; route by user language.

For agents that call tools, keep the tool schema in English (that's what APIs LLMs can use covers) but keep the reasoning and user-facing responses in the user's language.

dir="rtl" flips text. It doesn't fix your icons, your padding shortcuts, your date formats, or your table columns.

• Use logical CSS properties (margin-inline-start, not margin-left).
• Mirror directional icons (arrows, chevrons) — not calendars, not chart axes labeled with digits.
• Numbers in Arabic UI: decide once between Western Arabic (0–9) and Eastern Arabic (٠–٩); mixing is unprofessional.
• Date formats: Arabic locales expect Hijri as a secondary display in many contexts — offer both.
• Test with actual Arabic text; Lorem Ipsum RTL previews lie about line heights.

Public Arabic eval sets (MMLU-Arabic, ARC-Arabic) test general reasoning. They tell you almost nothing about your product's retrieval quality or generation quality on your domain.

Build a golden set of 100–500 real questions per dialect you support. Annotate expected answers, expected retrieved chunks, and dialect. Run this set on every model swap. Use the evaluation framework from evaluating LLMs and classical ML.

• Arabic tokenizes to ~1.5–2× more tokens than English for the same meaning — budget accordingly.
• Choose tokenizers thoughtfully: cl100k_base handles Arabic reasonably; older tokenizers explode it.
• For pure translation tasks, cheaper models often suffice; reserve premium models for reasoning.
• See LLM cost dashboard in BigQuery to attribute cost by user language.

Track separate metrics per language:

• Retrieval hit-rate per dialect group.
• Answer quality (human eval) per dialect group.
• Fall-through rate — how often the model responds in the wrong language.
• Escalation rate — how often users switch languages mid-conversation (signal that your system is failing).

Wire into your general observability stack from observability for AI systems.

Non-English inference is often served from fewer regions. Check your provider's Arabic-region latency; a Middle East user routed to US-East is 250ms of network before any compute.

For self-hosted models, one region in EU-Central + one in EU-South covers most of North Africa and the Levant acceptably. GCC users benefit from an ME-Central deployment.

1. Multilingual embedding model (BGE-M3 or Cohere v3).
2. Deliberate Arabic normalization at ingestion + query time.
3. Query rewrite between dialect and MSA.
4. Language-specific prompts routed by detected language.
5. Hybrid retrieval (dense + BM25) for mixed-script robustness.
6. Golden eval set per dialect, run on every deploy.
7. Metrics segmented by language, always.