MiniCPM5-2B: A 2.52B On-Device Model That Beats Qwen3.5-4B on Benchmarks
OpenBMB's MiniCPM5-2B scores 53.9 across 34 benchmarks, beats Qwen3.5-4B at 51.1, and runs on-device via Ollama, llama.cpp and LM Studio. Apache 2.0.
OpenBMB released MiniCPM5-2B on September 7, 2026, the second checkpoint in the MiniCPM5 series. The model packs 2,516,756,480 parameters into a standard Llama architecture with a 131,072-token context window, and averages 53.9 across 34 benchmarks. That puts it ahead of Qwen3.5-4B, which scores 51.1 despite being nearly twice the size. The weights are Apache 2.0 and load without custom code into vLLM, llama.cpp, Ollama, LM Studio, and several other mainstream engines.
What happened
| Detail | Value |
|---|---|
| Total parameters | 2,516,756,480 (2.52B) |
| Non-embedding parameters | 1,981,982,720 |
| Context window | 131,072 tokens |
| Architecture | LlamaForCausalLM, 42 layers, 16 query heads, 2 KV heads |
| Benchmark average (34 tasks) | 53.9 |
| Licence | Apache 2.0 |
| Inference engines supported | vLLM, SGLang, Transformers, llama.cpp, Ollama, LM Studio, MLX, FlagOS |
OpenBMB benchmarked MiniCPM5-2B against models in the same size class (LFM2.5-2.6B, Qwen3.5-2B, Gemma-4-E2B-it) and listed larger models for reference, including Qwen3.5-4B, granite-4.2-3B and LFM2.5-8B-A1B. The comparison is honest: OpenBMB flags which rows came from Artificial Analysis and which were reproduced internally.
Where MiniCPM5-2B wins and where it doesn’t
The story is not uniform across task types. Tool use and coding agents are where the model separates itself most clearly from the competition.
| Benchmark | MiniCPM5-2B | Best reference baseline |
|---|---|---|
| τ²-Bench Telecom (tool use) | 97.1 | , |
| BFCL v4 (tool use) | 66.6 | , |
| τ³-Bench Banking (tool use) | 20.8 | 6.8 (Qwen3.5-4B) |
| LiveCodeBench v6 (code) | 69.1 | 56.4 (Qwen3.5-4B) |
| SWE-bench Verified (code agents) | 46.4 | 33.6 (Qwen3.5-4B) |
| NoLiMa (long-context retrieval) | 68.1 | 43.5 (Qwen3.5-4B) |
| MMLU-Pro (general knowledge) | 70.8 | 78.0 (Qwen3.5-4B) |
| Humanity’s Last Exam | 8.9 | 9.9 (Qwen3.5-4B) |
| AA-LCR (long context) | 59.0 | 61.0 (Qwen3.5-4B) |
| LongBench v2 | 43.7 | 47.3 (Qwen3.5-4B) |
The pattern is clear: the model excels at structured tasks where a correct tool call or code patch can be verified, and it trails on broad factual knowledge. Businesses asking “which small model should I use for an agent that calls APIs or generates code?” have a strong answer here. Businesses that need reliable general Q&A should look elsewhere or use a larger model.
How was it trained?
OpenBMB used a four-stage pipeline built on its UltraData tiered data management method. First came base training with stable and decay phases. Then mid-training shifted the model toward its target data distribution. Post-training opened with 400 billion tokens of deep-thinking supervised fine-tuning (SFT), followed by reinforcement learning using the JustRL II algorithm to train specialist teacher models for math, code, agentic tasks and writing.
The final step is On-Policy Distillation (OPD), which merges 16 RL expert models, five of them focused on agentic work, into the single shipped checkpoint. Rather than building a new corpus, OPD reuses the RL prompts and computes a reverse KL divergence between the student and teacher outputs as the training signal. According to OpenBMB, the RL plus OPD stage adds 10.96 average points on reasoning and general benchmarks, and 6.96 points on agentic ones.
What data ships with the model?
OpenBMB released the full data stack alongside the weights: Ultra-FineWeb, Ultra-FineWeb-L3, UltraX, UltraData-Code, UltraData-Math, UltraData-SFT-2605, UltraData-SFT-Agent-2609 (500K agent samples), and UltraData-RL-2609 (over 80K RL samples). Intermediate checkpoints for the Base, Midtrain and SFT-only stages are also published, so the effect of each training phase can be measured independently rather than taken on faith.
Why it matters
Most on-device or edge deployments have had to accept a large gap between model capability and hardware reality. A 2.52B model that outperforms a 4B model on agentic tasks, runs without custom kernels, and drops into Ollama or LM Studio with no friction meaningfully changes that calculus. For teams building AI integrations that need to run locally (privacy, latency, or cost reasons), MiniCPM5-2B is now a serious candidate.
The open data release is the part that gets underreported. Releasing training corpora and intermediate checkpoints lets other researchers verify or replicate the RL plus OPD gains. That kind of transparency is still rare among labs shipping capable small models, and it makes the benchmark claims far more credible than a table with no methodology attached.
Our take
The benchmark headline is genuinely surprising. Beating Qwen3.5-4B at half the size on a 34-task aggregate is not a marketing move you can fake without someone noticing. The specific gaps matter though. If your use case is a coding or tool-calling agent, the numbers here are real and significant. If you need the model to answer broad factual questions accurately, the MMLU-Pro deficit (70.8 vs. 78.0) and the near-zero Humanity’s Last Exam score (8.9) tell you this is not that model.
For agencies and developers building on-device agents, particularly for automation pipelines or structured API-calling workflows, this is worth testing this week. The Ollama and LM Studio support means the barrier to a local proof of concept is essentially zero. We cover the broader space of small agentic models regularly in our AI news coverage, and MiniCPM5-2B stands out as one of the more honest releases in that category.
What to do about it
- Pull the model via Ollama or LM Studio and run your own tool-calling or code-generation prompts before committing to any benchmark comparisons.
- Check the intermediate checkpoints on Hugging Face if you want to understand what the SFT-only baseline looks like versus the full RL plus OPD pipeline.
- If you are building an agentic workflow that needs to run locally or at the edge, put MiniCPM5-2B on your shortlist alongside Qwen3.5-2B and test both against your actual task.
- If general knowledge accuracy is critical for your use case, test MMLU-Pro-style prompts before deploying, because the gap to larger models is real.
The Apache 2.0 licence means you can use it commercially without asking permission, which removes one common blocker for production deployment.
Frequently asked questions
How does MiniCPM5-2B compare to Qwen3.5-4B?
MiniCPM5-2B averages 53.9 across 34 benchmarks, beating Qwen3.5-4B's 51.1, despite being roughly half the size. The advantage is strongest on tool use and coding tasks. Qwen3.5-4B leads on general knowledge benchmarks like MMLU-Pro (78.0 vs 70.8).
Can MiniCPM5-2B run locally on a laptop?
Yes. The model uses a standard Llama architecture and is supported by Ollama, LM Studio, llama.cpp and MLX, all of which run on consumer hardware without custom kernels. The weights are available under an Apache 2.0 licence.
What is On-Policy Distillation (OPD) in MiniCPM5-2B?
OPD is the final training stage where 16 specialist RL teacher models are merged into one student model. The student is trained using the RL prompts as distillation data, with reverse KL divergence as the training signal rather than a separate verification system. OpenBMB reports this stage adds about 11 points on reasoning benchmarks.
What licence is MiniCPM5-2B released under?
Apache 2.0, which allows commercial use without a separate licence agreement. Training datasets and intermediate checkpoints are also publicly released.
