The Chinese dialect tier list nobody benchmarks: Cantonese, Wu, Hokkien, minority languages and code-switching

Every public multimodal benchmark tests standard Mandarin — studio-grade audio, broadcaster delivery, written-register vocabulary. Not one mainstream benchmark tests Cantonese, Wu, Min Nan, Sichuan-Chongqing Mandarin, or Uyghur and Tibetan; still less does any test the “mixed Chinese-English cross-border call-centre accent”. Yet these are exactly the sounds Chinese models handle every day in real business.

This essay builds a separate mini tier list: eight classes of Chinese dialects and accents that nobody tests, assessed model by model across 32 models. It shares only one model with the top three of the 2026 China native multimodal LLM tier list.

Gate one: only 11 of 32 can hear sound

Before discussing how good a model is at dialects, there is a structural gate to pass: only 11 of the 32 models have an audio modality. The other 21 do not compete in dialect scenarios at all — not because they score low, but because they cannot receive sound.

With an audio modality (11)	Without an audio modality (21)
Doubao Seed 2.0 Pro, Qwen3.5-397B, Qwen3.8-Max, Qwen3-Omni-30B, MiMo-V2.5, MiniMax M2.5, Step-3, LongCat-Next, Spark X2, WITA-Omni, Zidong Taichu 2.0	Kimi K3, GLM-5, GLM-5.3, Qwen3.8-Flash, DeepSeek-V4.1-Flash, DeepSeek-OCR 2, SenseNova 6.5 Pro, ERNIE 5.1, Keye-VL-2.0, Hunyuan-Vision, Pangu 5.5, InternVL 4.0 and others

Half of these 11 can receive sound but cannot make out accents: their audio channels are designed mainly for Mandarin and English, and dialects are covered only as noise under robustness. Across the whole site, exactly three treat dialects as a selling point.

Gate two: three capability tiers, not one

“Supports dialects” is a badly compressed claim. The real speech pipeline has three tiers, and passing the first does not mean passing the second:

• L1 transcription (ASR): turning dialect speech into text. Most models with audio can pass this tier.
• L2 understanding: grasping the slang, word order and implication inside a dialect, rather than doing phoneme mapping alone. Cantonese “你食咗饭未”, for instance, must be understood as “have you eaten”, not transcribed character by character.
• L3 generation: producing reply text in dialect register, or synthesising dialect speech. Almost nobody passes this tier — no model gets both L1 and L3 right on dialects.

This table scores L1 and L2 only. L3 is broken out in its own section, because it is a different product (speech synthesis), not an understanding model’s capability.

The eight classes and their difficulties

Class	Core difficulty	Why public benchmarks do not test it
Cantonese (including Hong Kong colloquial)	Six tones, lazy sounds, heavy English borrowing	Jyutping annotation is costly, and mainland business demand is unevenly distributed
Min Nan / Taiwanese	Tone system vastly different from Mandarin; weak lexical correspondence	Training corpora scarce, and cross-strait context is sensitive
Wu (Shanghainese / Suzhou)	Tone sandhi, voiced consonants, large internal variation	Speakers are mostly middle-aged and elderly; few public recordings
Sichuan-Chongqing Mandarin	Fast speech, heavy erhua, high mutual intelligibility with Mandarin	Too “close to Mandarin”; benchmarks see no need to test it separately
Northeastern Mandarin	Vocabulary differs more than phonology	Treated as a dialect variant of Mandarin
Central Plains Mandarin (Henan / Shaanxi)	Jian-tuan distinctions, residual entering tone	As above, blurred into “Mandarin robustness”
Minority languages (Uyghur / Tibetan / Mongolian)	Non-Sinitic families; need separate phonemes and lexicons	High data sensitivity; no public evaluation set
Mixed Chinese-English (code-switching)	Intra-sentence code-switching; accent degrades English recognition	Benchmarks are designed monolingually; code-switching is out of scope

Classes four, five and six share one problem: “too much like Mandarin”. Benchmarks count them under Mandarin robustness by default, so they never get an independent score. Classes seven and eight share another: “the data cannot be public” — the same structural reason as the surveillance-video mini tier list.

Per-class top three

Each class lists its top three. Scores are 0–100 desk assessments based on vendor dialect cases, comparable published metrics and community reproductions; evidence grades are marked item by item in the text.

Cantonese (including Hong Kong colloquial)

1. Spark X2 92 — dialect ASR is its public selling point; first in China on speech-semantics share
2. Doubao Seed 2.0 Pro 90 — strong Chinese everyday-life priors; good Cantonese slang understanding
3. Qwen3.8-Max 88 — solid general audio channel; occasional dropped characters on lazy sounds

Wu (Shanghainese / Suzhou)

1. Spark X2 86 — best handling of tone sandhi
2. Qwen3.8-Max 80 — strong generalist; average on internal dialect variation
3. Doubao Seed 2.0 Pro 78 — corpus accumulated in the Yangtze River Delta; weaker on Suzhou than Shanghainese

Min Nan / Taiwanese

1. Spark X2 84 — the only model that reaches L2
2. Doubao Seed 2.0 Pro 76 — L1 usable; slang starts to drop
3. Qwen3.8-Max 74 — strongest at its price, but still mainly phoneme transcription

Sichuan-Chongqing Mandarin

1. Spark X2 90
2. Doubao Seed 2.0 Pro 88
3. Qwen3.8-Max 86

Northeastern Mandarin

1. Spark X2 91
2. Doubao Seed 2.0 Pro 88
3. Qwen3.8-Max 85

Central Plains Mandarin (Henan / Shaanxi)

1. Spark X2 90
2. Doubao Seed 2.0 Pro 86
3. Qwen3.8-Max 84

Minority languages (Uyghur / Tibetan / Mongolian)

1. Spark X2 82 — product accumulation in minority-language speech for education scenarios
2. Zidong Taichu 2.0 72 — academic corpus; method reproducible but weakly productised
3. The other 9 models with audio all fall below 70, and most cannot give stable L1 transcription

Mixed Chinese-English (code-switching)

1. Doubao Seed 2.0 Pro 90 — e-commerce and cross-border priors; accurate switch-point detection
2. Qwen3.8-Max 88 — good multilingual foundation; English segments lose points under heavy accent
3. Qwen3.5-397B-A17B 86 — the 32-language OCR corpus advantage

The common thread in the failure modes: the dialect problem is not “can’t hear clearly” but “heard clearly and understood wrongly”. There is a 10–20 percentage-point gap between transcription accuracy and semantic-understanding accuracy, and the great majority of evaluations report only the former.

The dialect mini tier list: Spark rises from the top 20 to 1st

Weighting the eight classes by business frequency (Cantonese 18%, Sichuan-Chongqing 15%, mixed Chinese-English 14%, Northeast 12%, Wu 12%, Min Nan 12%, Central Plains 10%, minority languages 7%):

Rank	Model	Dialect weighted	Overall rank	Gap
1	Spark X2	86.5	18	+17
2	Doubao Seed 2.0 Pro	84.4	2	0
3	Qwen3.8-Max	83.2	1	−2
4	Qwen3.5-397B	81.2	3	−1
5	WITA-Omni	77.4	23	+18
6	MiMo-V2.5	76.5	7	+1
7	Step-3	74.2	17	+10
8	MiniMax M2.5	72.2	19	+11
9	Qwen3-Omni-30B	70.0	10	+1
10	LongCat-Next	66.4	9	−1
11	Zidong Taichu 2.0	65.5	25	+14

Three things happen at once:

First, the model ranked 18th overall is 1st here, 2.1 points ahead of 2nd. This reproduces the rank-divergence index phenomenon again: overall rank has near-zero predictive power for vertical speech scenarios. Spark’s composite is only 84.5, because its vision, video and agent scores are not first-tier; but on “understanding what people actually say”, it is the only model on the site that treats dialects as a primary training target.

Second, WITA-Omni rises from 23rd to 5th. Its input is a continuous audio-visual stream — a service robot has to judge “who is speaking, and whether to respond” in noisy, accent-diverse environments. This kind of “interaction-scenario audio” and “content-scenario audio” are two different capabilities, and the overall board flattens them together.

Third, only one of the overall top three stays in the dialect top three (Doubao). Qwen3.8-Max drops from 1st to 3rd — not because it is weak, but because “Mandarin is enough” stops working in dialect scenarios.

L3 generation: a tier almost nobody passes

If “replying in dialect” is also counted as a capability, the picture is completely different: of the 11 models with audio, only two can generate dialect-register text (Spark, Doubao), and both are stable only on Cantonese; not one model on the site can synthesise dialect speech.

This means any “dialect customer service” product on the market today is in fact a three-part assembly — “L1/L2 understanding + a Mandarin corpus + third-party TTS” — not end-to-end model capability. When procuring, ask plainly: are you buying “understanding dialect” or “speaking dialect”? The latter currently requires separately procured speech synthesis and sits in no multimodal model’s scoring.

The acceptance checklist: do not accept dialects on Mandarin samples

Do not accept-test on public leaderboard screenshots. Pick 60 clips of your own audio, in this ratio:

• 20 typical dialect clips (target dialect, normal accent, quiet environment)
• 15 accent-drift clips (locals long away from home, Mandarin words mixed in)
• 10 mixed Chinese-English clips (the real recording shape of cross-border and trade call centres)
• 10 noisy-scene clips (in-store, roadside, in-car)
• 5 “should refuse” clips (blurred, overlapping speakers, unrecognisable language)

The last group matters most. A good model says “I can’t hear you, please repeat”; a bad model guesses a Mandarin homophone. Record at least three metrics: transcription accuracy, semantic-understanding accuracy and refusal correctness; an evaluation that reports only the first systematically overstates dialect capability.

Boundaries

• Eleven entrants is a structural limit, not a capability ranking. The absence of an audio modality keeps 21 models out of this table — itself an instance of modality negative space.
• Scores come from desk assessment, not a closed laboratory. The basis is vendor dialect cases, comparable published metrics and community reproductions; there is no third-party, uniformly reproduced dialect evaluation set.
• Dialects are a continuum, not discrete classes. Wu varies enormously internally, and representing a class with one score loses information; Cantonese likewise splits into Guangfu, Siyi and Gaoyang.
• Weights move ranks. Raise Cantonese’s weight from 18% to 30% and the gap between Spark and Doubao narrows, while WITA falls out of the top five.

Reproduction

The eight-class scores come from this site’s desk-assessment standard; the weighted table and the overall-rank gaps are computed directly from the site composite (composite = Σ dimension score × weight). You can change the weights and re-sort to match your own business distribution.

If this essay leaves only one sentence: do not select for dialect scenarios off the overall board; first confirm whether your model can receive sound at all, then confirm whether it understands or merely hears.