Three things to remember
- The agent dimension has a three-way tie at 95 — Kimi K3, Qwen3.8-Max and GLM-5.3. Their failure-recovery rates are 64%, 71% and 78%: one joint leader sits fourteen points below another. Public benchmarks stop discriminating at this score level.
- Of six failure modes, permission-dialog deadlock and back-navigation storms are the shared weakness of Chinese models — and appear in almost no public benchmark.
- The three models whose P95 coordinate error crosses 2% of screen width (about 21.6 physical pixels at 1080px, below the mobile minimum tap target) are all mid-to-low price: MiMo-V2.5, Qwen3.8-Flash and DeepSeek-V4.1-Flash. Their problem is not slightly weaker capability but unacceptable tail risk.
A three-way tie at 95, and one metric nobody tests
The agent dimension of the 2026 China native multimodal LLM tier list contains the only three-way tie on the entire board: Kimi K3, Qwen3.8-Max and GLM-5.3, all at 95.
The tie is not a coincidence — it is a ceiling effect. Every public GUI/tool-use benchmark in circulation — AndroidWorld, OSWorld, ScreenSpot, τ-bench — stops discriminating in this score band. One 95 and another 95 could be “wrong once in ten but recovers every time” versus “wrong once in ten and then taps the same button forever”. Those two models differ by an order of magnitude in production usability, yet look identical on a tier list.
So we did not test success rate. Every vendor publishes success rate; testing it is paraphrasing press releases. We tested what happens after failure.
One clarification first: the agent dimension is not the GUI dimension
Of the 32 models here, only four explicitly list a GUI-control modality: Kimi K3, Qwen3.8-Max, GLM-5.3 and Qwen3.8-Flash. For the other 28, the agent score covers tool use, function calling, multi-step planning and structured output in general — they can write a function call, but they cannot necessarily tap a button.
None of this cycle’s eight new entries is on the declared list, for several different reasons: InternVL 4.0 covers images and text only (no GUI training data; agent 78); NeuroXess-N1 ships as an in-hospital project delivery (agent 76, no phone automation in the deliverable); and the six industry models (Guanlan 68, Congrong 70, Zidong Taichu 70, Ling 2 80, Yanxi 2 78, Fuxi 2 75) all sit below the test threshold on agent and ship no phone automation at all — security vendors deliver cameras and boxes, scenario models deliver industry solutions. None enters the test set below — not because their scores are too low, but because what they sell does not include this capability.
This distinction matters because it means: the models ranked 4th through 8th on agent in the 2026 China native multimodal LLM tier list (Doubao Seed 2.0 Pro 93, Qwen3.5-397B 92, GLM-5 91, MiMo-V2.5 90, LongCat-Next 88) will fail outright at phone automation — not a matter of scoring a bit lower, but of not having the modality at all.
The reverse holds too: Qwen3.8-Flash’s agent score is only 87, 13th overall, yet it is one of the four genuine GUI models — and the only one with open weights (Apache-2.0). In the sub-class of “GUI agents you can deploy yourself”, it is the only option. There is no second place.
Our test set is therefore 10 models: the four that declare GUI plus the top eight on agent among those that do not. The latter represent “doing GUI with general capability”, and that comparison is itself part of the finding.
Failure recovery rate: six failure modes, only counting walk-backs
We defined six failure modes that are frequent in real automation flows and almost entirely uncovered by public benchmarks:
| Failure mode | Trigger | Typical symptom |
|---|---|---|
| Tap into void | Target covered by an overlay / animation unfinished | The tap lands on the mask; no state change |
| Mis-tap | Dense adjacent controls (lists, toolbars) | Enters the wrong page while believing the task progressed |
| Loop | Repeated page structure (feeds, table pagination) | Repeats the same action with no awareness of a step ceiling |
| Cross-app amnesia | Switch from app A to app B and back | Loses the previous app’s context and starts over |
| Permission-dialog deadlock | System dialogs (location, notifications, privacy) | Reads the dialog as page content, or retries “OK” forever |
| Back-navigation storm | Tapping back repeatedly to recover state | Retreats to the home screen; the task is lost |
Each mode has 20 constructed samples — 120 failure scenarios in total. The recovery bar is strict: with no external hint, the model must itself notice that the current state does not match the intended one and return to the correct path within five steps. Eight steps does not count; noticing and then quitting with a report counts as half — and we do not count halves, only full recoveries.
| Rank | Model | Agent score | Overall | Failure recovery rate | Declares GUI |
|---|---|---|---|---|---|
| 1 | GLM-5.3 | 95 | #4 | 78% | Yes |
| 2 | Qwen3.8-Max | 95 | #1 | 71% | Yes |
| 3 | Doubao Seed 2.0 Pro | 93 | #2 | 69% | No |
| 4 | Qwen3.5-397B | 92 | #3 | 66% | No |
| 5 | Kimi K3 | 95 | #5 | 64% | Yes |
| 6 | GLM-5 | 91 | #6 | 61% | No |
| 7 | MiMo-V2.5 | 90 | #7 | 58% | No |
| 8 | LongCat-Next | 88 | #9 | 52% | No |
| 9 | Qwen3.8-Flash | 87 | #13 | 47% | Yes |
| 10 | DeepSeek-V4.1-Flash | 85 | #12 | 41% | No |
The most important row is row five. Kimi K3 ties for first on agent (95) yet has the worst recovery rate of the three joint leaders at 64% — fourteen points below GLM-5.3. Read only the tier list’s agent column and you would think the three are interchangeable; run unattended long-flow automation and you have a one-in-three chance of picking right.
The second most important rows are three and four: Doubao Seed 2.0 Pro and Qwen3.5-397B do not declare GUI modality, yet their recovery rates rank 3rd and 4th — above Kimi K3, which does. Recovery comes mostly from general state reasoning and self-checking, not from GUI-specific training. That is a counter-intuitive result, and the reason we do not recommend “supports GUI” as the only filter.
Third are the last two rows. The two new Flash-class models — Qwen3.8-Flash (47%) and DeepSeek-V4.1-Flash (41%) — fall off a cliff below 50%. That is no coincidence: recovery requires holding “expected state” and “actual state” in context simultaneously and comparing them, which is a job for inference length and depth. The first thing a Flash tier cuts to lower unit price is exactly that.
Coordinate error P50 / P95: predicts production incidents better than success rate
The second metric is click-coordinate error as a percentage of screen width. We use P95 rather than the mean, because the mean is meaningless in automation — your flow does not break under average conditions, it breaks on the 20th tap.
| Model | Coordinate P50 | Coordinate P95 | Crosses the 2% threshold? |
|---|---|---|---|
| Qwen3.8-Max | 0.4% | 1.1% | — |
| GLM-5.3 | 0.5% | 1.3% | — |
| Kimi K3 | 0.5% | 1.6% | — |
| Doubao Seed 2.0 Pro | 0.6% | 1.5% | — |
| Qwen3.5-397B | 0.6% | 1.7% | — |
| MiMo-V2.5 | 0.8% | 2.1% | ✗ Over |
| Qwen3.8-Flash | 0.9% | 2.4% | ✗ Over |
| DeepSeek-V4.1-Flash | 1.0% | 2.8% | ✗ Over |
The 2% threshold is not our invention. On a 1080×2400 phone, 2% of screen width is about 21.6 physical pixels — already under two-thirds of the mainstream mobile minimum tap target (about 30 physical pixels under the 44×44 pt design guideline). A P95 above 2% means: at least one tap in twenty lands outside the target control. For dense lists, toolbars and small icons, that error rate is simply a mis-tap.
Three of eight cross the line, all mid-to-low price. That agrees exactly with the recovery-rate conclusion and corroborates it: for Flash-class models in GUI scenarios the problem is not “slightly weaker capability” but “unacceptable tail risk”.
Note that Qwen3.8-Max is first on this table (P50 0.4%, P95 1.1%) and second on recovery rate. Read together, it is the only model that is top-two on both coordinate precision and recovery. That is why it holds first overall on the 2026 China native multimodal LLM tier list, and why among the three-way agent tie it is the one that survives follow-up questions.
Combining the two metrics into a selection rule
Do not read a single number. Use this matrix:
| Recovery ≥ 65% | Recovery < 65% | |
|---|---|---|
| P95 ≤ 2% | Unattended long flows: GLM-5.3, Qwen3.8-Max, Doubao Seed 2.0 Pro, Qwen3.5-397B | Supervised short flows: Kimi K3 |
| P95 > 2% | (no model falls here in this test) | Prototype validation only, never production: MiMo-V2.5, Qwen3.8-Flash, DeepSeek-V4.1-Flash |
Three implementation notes:
First, hard-code the step ceiling for unattended flows. Even GLM-5.3 at 78% recovery cannot save 22% of failures. The engineering answer is not to find a 100% model (none exists) but to make that 22% fail cheaply: step ceilings, state snapshots, breakpoint replay. Model selection decides how much fallback code you write, not whether you write it.
Second, Kimi K3 is the most misread model on this table. Agent 95 tied for first, recovery 64% fifth, coordinate P95 1.6% passing. It lands in the “supervised short flows” cell — which is exactly what most demo scenarios are, and why it scores well on public benchmarks. Benchmarks test short flows; production runs long ones.
Third, if you need a privately deployed GUI agent, there is one option and it does not qualify. Qwen3.8-Flash is the only open-weights model among the four GUI declarers, but its recovery is 47% and P95 is 2.4% — both fail. That means self-hosted phone automation had no usable solution as of October 2026 — not a pricing question, a capability question. We meet this again in the enterprise selection playbook’s veto-gate board.
Boundaries and method
Failure recovery rate and coordinate error are this site’s own desk assessments, not closed-lab reproductions: 120 failure scenarios, roughly 800 taps per model, three real devices plus two emulators. The numbers support relative ordering only; absolute values are not guaranteed across environments. For the four tested models that do not declare GUI modality, coordinates are produced indirectly through general visual grounding, so their error readings are expected to run high.
The full rubric and evidence criteria are on the methodology page; every agent score in this essay can be re-weighted in the compare lab.
FAQ about this essay
How is recovery rate measured?
We build 20 samples for each of six failure modes — 120 scenarios in total: tap-into-void, mis-tap, loop, cross-app amnesia, permission-dialog deadlock and back-navigation storm. The bar is strict: with no external hint the model must itself notice that the current state does not match the intended one and return to the correct path within five steps. Eight steps does not count, and neither does noticing then giving up and reporting. Recovery rate is the share of scenarios fully recovered.
Do success rates still matter?
Yes, but as a gate rather than a sort key. Below a success threshold a model is eliminated; above it, recovery rate and P95 coordinate error produce an ordering that matches real-device behaviour. The tier list’s agent dimension is composed in exactly that order.
