InclusionAI has released LLaDA-UI, an approximately 16.7-billion-parameter vision-language model that uses block-wise diffusion to predict actions in mobile, desktop and web interfaces. The open model is notable less for a leaderboard claim than for a different way of generating GUI actions.
The official LLaDA-UI model card publishes BF16 safetensors, a Transformers inference path and an SGLang path. The official code repository provides the model-specific serving code. A linked research paper describes the architecture.
The short version
- LLaDA-UI is a roughly 16.7B-parameter mixture-of-experts GUI agent.
- It uses block-wise diffusion rather than purely left-to-right action generation.
- It accepts native-resolution visual input through a ViT initialized from SigLIP with 2D RoPE.
- It returns text reasoning and structured actions with coordinates normalized to a 0-999 grid.
- Its SGLang route requires model-specific patches, not an unmodified stock server.
Block-wise diffusion changes the action loop
An autoregressive agent commits to one token after another. A diffusion language model begins with a noisy or masked block and refines it across several steps. LLaDA-UI applies that process to a block containing the reasoning and action representation.
The reference example uses a generation length of 32, 32 diffusion steps and a block length of 32. Those settings are not a universal optimum. They show the basic contract: the model repeatedly denoises a bounded action block until it produces a structured result.
Native-resolution vision preserves interface geometry
GUI agents fail when they lose the relationship between what they see and where it appears. LLaDA-UI’s visual encoder is designed for native-resolution inputs and uses two-dimensional rotary position embeddings. That preserves spatial structure more directly than forcing every screenshot into one fixed square.
The model emits coordinates on a normalized 0-999 grid. A point such as [742,186] maps to 74.2% of the screen width and 18.6% of the screen height. The application must convert that normalized point into the target device’s pixel coordinates before executing a tap or click.
The action format is designed for execution
LLaDA-UI can produce text reasoning and structured GUI actions. The model card also reserves [-1,-1] for cases where a requested target is infeasible or unrelated. That sentinel matters because an agent needs a way to decline an unsafe coordinate rather than always inventing one.
Execution still needs an external controller. The model proposes an action; a trusted automation layer should validate the schema, translate coordinates, enforce permissions and confirm the resulting state. Our computer-use agent safety guide explains why browser and shell permissions should remain explicit.
The checkpoint is about 32 GB before runtime
The model card describes an approximately 32 GB BF16 checkpoint. That is only the stored weight footprint. Visual tokens, activations, diffusion steps and the serving framework require additional GPU memory. A 32 GB checkpoint should not be read as a 32 GB end-to-end memory requirement.
| Layer | Published requirement | Operational implication |
|---|---|---|
| Checkpoint | About 32 GB BF16 | Storage is only the starting point. |
| Reference stack | Python 3.10, PyTorch 2.5.1, Transformers 4.51.0 | Version pinning reduces compatibility surprises. |
| Attention kernel | FlashAttention 2.7.4.post1 | GPU and build compatibility need validation. |
| SGLang | Model-specific patches | Stock SGLang alone will not start the model. |
A practical deployment sequence
First reproduce the Transformers example on a known screenshot. Next validate coordinate conversion without allowing real clicks. Then add schema checks and an allowlist of permitted actions. Only after replay tests pass should the agent control a disposable browser or emulator.
For the SGLang path, use the repository’s patched implementation and record the commit. The same discipline applies to the agent harness. Persistent context can improve a GUI agent, but it can also preserve stale or sensitive state; our agent context guide covers that tradeoff.
What to measure before trusting it
A useful evaluation should separate perception, grounding and execution. Measure whether the model identifies the correct control, whether its normalized point lands inside the control and whether the action produces the intended state. Also record recovery behavior after pop-ups, layout changes and failed actions.
Do not compress those stages into one success rate. A model can understand the instruction but miss the coordinate, or select the right coordinate while the automation layer executes the wrong action.
What the release does not yet prove
The accessible primary sources document architecture and deployment, but they do not by themselves establish broad superiority over production GUI agents. MustHave.ai is therefore not presenting a cross-model winner. The strongest claim supported today is that LLaDA-UI offers an open, inspectable diffusion approach to structured interface actions.
Primary sources and disclosure
- InclusionAI: LLaDA-UI model card and weights
- InclusionAI: LLaDA-UI code repository
- LLaDA-UI research paper
Checked September 16, 2026. Architecture and runtime details come from InclusionAI’s model card, repository and paper. MustHave.ai has not independently reproduced the checkpoint or evaluated live-device reliability.
