Baidu has made the ERNIE 5.5 Flash model available for download

Baidu’s ERNIE team made the weights for the ERNIE-5.5-Flash-Base model available for download on November 5, 2026. The text-only model uses a 280-billion-parameter mixture-of-experts (MoE) architecture, activating 14 billion of those parameters for each token.

The released file is a “base” version: pretraining is complete, but intermediate training and instruction tuning have not yet been done. Baidu says this gives developers a shared starting point for domain adaptation, instruction tuning, and reinforcement learning experiments. The company says the trained ERNIE 5.5 Flash version will be built on top of this foundation.

Compressed attention for long context

According to the model card, the model has 43 layers, with a hidden size of 4096 and 64 attention heads. In 37 of those layers, it uses “Heavily Compressed Attention”: the cache keeps a single entry for each 128-token block, with a short last-token window added on top. The remaining six layers preserve token-level access with Multi-head Latent Attention and use an indexer to choose which older positions to read. According to Baidu, this sparse selection keeps the core attention cost per query constant as context grows.

Every layer except the first includes an MoE block: 512 routed experts, 10 of them active, plus one shared expert. The experts do not operate at the model’s full hidden width, but instead in a 2048-dimensional latent space. Baidu says this choice roughly halves expert routing traffic and computational load.

33.5 trillion tokens and 1 million-token inference

Pretraining was done on 33.5 trillion tokens using the Muon optimization algorithm and FP8 mixed precision. During training, the context window was gradually expanded to 128K tokens; in fine-tuning and inference, the model supports context up to 1 million tokens. The dataset includes multilingual web content as well as code and software development records: commits, issues, pull requests, code reviews, generated tests, and fixes that passed CI. These choices push the model closer to repository-level reasoning and agent-based software engineering scenarios.

Baidu also released its FlashMask attention kernels. According to the company’s measurements, on Blackwell (SM100) hardware they deliver speedups of up to 175 percent in the forward pass and 74 percent in the backward pass. The weights are available in FP8 and BF16 safetensors format, and can be served with SGLang and vLLM.

Ahead in its own tests, behind in some others

Baidu compared the model against open checkpoints such as DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base under the same test setup. In five-shot MMLU, ERNIE scored 93.3, while DeepSeek came in at 89.4, Hy3 at 87.5, and MiMo at 87.6. ERNIE also led HumanEval+ with 82.3. But the table does not show across-the-board dominance: on FullStackBench, Hy3 scored 56.7 versus ERNIE’s 54.6, and on InfiniteBench, DeepSeek’s 48.0 was ahead of ERNIE’s 46.5. So the results vary by task.

The license limits geography and revenue

The model’s terms of use are quite different from typical open-source expectations. The ERNIE 5.5 Flash Base license excludes the European Union, the United Kingdom, and South Korea; use, modification, and distribution are not authorized in those regions. There is no charge for non-commercial use of the model. Commercial users with annual revenue below $50 million can proceed with a one-time notification and a prominent “Powered by ERNIE” attribution; those above that threshold must obtain separate written permission from Baidu. The license also prohibits using the model or its outputs to develop another AI system. That is a significant restriction for teams interested in distillation or generating synthetic data.

Full-model fine-tuning is no lightweight job either: NVIDIA’s Megatron Bridge recipe recommends 16 nodes with four GB300 GPUs per node. In short, ERNIE 5.5 Flash Base is a solid reference point for researchers who want to compare training methods, but its geographic and commercial limits narrow practical access.

Source: Unite.AI