Baidu released ERNIE 5.5 Flash Base on November 5, an open-weight sparse mixture-of-experts language model with 280 billion total parameters and 14 billion activated per token. Baidu says the checkpoint was pre-trained from scratch on 33.5 trillion tokens and is the model as it stood at the end of pre-training, before any mid- or post-training.
The text-only model was trained at a 128K context length, and the released configuration enables a context window of 1 million tokens for fine-tuning and inference. Baidu published both FP8 and BF16 weights on Hugging Face and ModelScope under its ERNIE 5.5 Flash Base Model License Agreement v1.0.
Baidu compared the model with three other base models of similar size, all run through the same evaluation pipeline, prompts and few-shot settings. On MMLU-Pro, ERNIE 5.5 Flash Base scored 78.4, against 69.2 for DeepSeek-V4-Flash-Base, 66.9 for Hy3-preview-Base and 65.7 for MiMo-V2.5-Base. It also led on MMLU at 93.3, GPQA at 60.2 and SimpleQA at 34.1.
Key facts
- 01Baidu released ERNIE 5.5 Flash Base on November 5 with FP8 and BF16 weights on Hugging Face and ModelScope.
- 02The sparse MoE model has 280B total parameters and 14B activated per token.
- 03It was pre-trained from scratch on 33.5T tokens at 128K context; the released config enables 1M context.
- 04Baidu's evals put it at 78.4 on MMLU-Pro versus 69.2 for DeepSeek-V4-Flash-Base.
- 05It ships under the ERNIE 5.5 Flash Base Model License Agreement v1.0.
The coding numbers show a similar gap. Baidu reports 82.3 on HumanEval+ versus 69.5 for DeepSeek-V4-Flash-Base, 81.5 on MBPP+ versus 67.2 for the same DeepSeek model, and 40.1 on LiveCodeBench, where the other three models scored between 31.9 and 33.5.
ERNIE 5.5 Flash Base did not top every row. DeepSeek-V4-Flash-Base scored higher on ChineseSimpleQA, 75.3 to 73.0, and also led on INCLUDE-lite, HELMET-Rerank and InfiniteBench. Hy3-preview-Base came out ahead on chain-of-thought BBH and FullStackBench.
Under the hood, the model has 43 layers, one dense and 42 mixture-of-experts, with 512 routed experts per MoE layer using top-10 routing plus one shared expert. Baidu lists a Multi-Router Latent MoE design, a Heavily Compressed Attention scheme that caches one entry per 128 tokens, and a module called ECHO in place of multi-token prediction. Pre-training used the Muon optimizer.
Baidu also published a FlashMask-4 attention kernel used in training, which it says runs up to 175% faster on the forward pass and 74% faster on the backward pass than FlashAttention-4 MaskMod on SM100 hardware. The base model can be served through SGLang and vLLM via the completions endpoint.
The release follows Baidu's ERNIE 4.5 open model family, announced in June 2025, and its 3B-parameter Unlimited-OCR-2 model, released on Hugging Face on October 27.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
