657MB Local Thinking Model: MiniCPM5-1B Fine-Tuned on Claude Traces

Community developer GnLOLot has published MiniCPM5-1B-Claude-Opus-Fable5-Thinking, a 1-billion-parameter model that runs entirely on local hardware with no API key or cloud calls required. Built on OpenBMB's MiniCPM5-1B base and fine-tuned on output traces generated by Claude, the model ships as GGUF files compatible with llama.cpp, Ollama, LM Studio, and similar runtimes. The smallest quantization weighs about 657MB. There are no published benchmarks, and the licensing situation around training on Claude outputs is unresolved.

What happened

Detail	Value
Base model	OpenBMB MiniCPM5-1B (1.08B parameters)
Architecture	LlamaForCausalLM, 24 layers, grouped-query attention
Context window	128K tokens (131,072)
Smallest GGUF (Q4_K_M)	~657MB
Recommended GGUF (Q8_0)	~1.1GB
Largest GGUF (F16)	~2.1GB
Compatible runtimes	llama.cpp, Ollama, LM Studio, jan, KoboldCpp
Benchmarks published	None

GnLOLot’s model keeps the thinking template that already exists in the MiniCPM5-1B base. That means you get two modes toggled by an enable_thinking flag: Think mode and No Think mode. In Think mode the model can emit a reasoning block before its final answer, which downstream apps can strip if needed. Recommended sampling settings for Think mode are temperature 0.9 and top_p 0.95.

The fine-tune targets coding and instruction following, according to the model card. The GGUF repository describes it as “post-trained on Fable 5 data,” referring to conversations generated by Claude Opus.

What this build method actually does (and does not do)

The source article is careful about terminology, and so should anyone evaluating this model. This is not classical distillation. Classical distillation compresses a teacher model by transferring signal from its weights or output probability distributions (logits). Claude’s weights and logits are not publicly accessible, so that path is closed.

What GnLOLot did instead: generate many conversations using Claude, capture the replies and reasoning traces as text, then run supervised fine-tuning on MiniCPM5-1B using those captured outputs as training data. The model learns to imitate Claude’s response format and style. It does not inherit Claude’s underlying reasoning capability.

OpenBMB’s own base model uses a documented on-policy distillation stage between its own internal teacher and student checkpoints. That is a different and more rigorous process than what is described here.

The practical ceiling is the parameter budget. A 1B parameter model cannot hold the knowledge or reasoning depth of a frontier model regardless of what data it trains on. Expect usable format imitation, not frontier-level output.

How to run it

The GGUF card documents a one-line Ollama command:

ollama run hf.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF:Q4_K_M

The same repository documents equivalent setup steps for llama.cpp, LM Studio, jan, and KoboldCpp. For most local setups, Q8_0 at around 1.1GB is the maintainer’s recommended default. Q4_K_M at roughly 657MB is the smallest available quantization, not the default, so the “657MB model” framing in headlines is technically accurate but slightly misleading.

Why it matters

Models this small that run offline have genuine uses: air-gapped environments, edge devices, low-latency local tools, and privacy-sensitive workflows where sending data to a cloud API is not acceptable. A 128K context window in a sub-1GB package is a legitimate capability for summarizing long documents without a subscription.

The broader trend is also worth tracking. Community developers are now routinely fine-tuning open-weight small models on outputs from larger proprietary models. Some businesses are exploring similar approaches to customize behavior for narrow tasks. If you are evaluating AI integration for internal tooling, local fine-tuned models are becoming a real option. For a fuller picture of what this kind of work looks like in practice, our AI integration service covers both cloud and local deployment paths.

What are the open questions?

Two issues are worth flagging before anyone depends on this model:

• No benchmarks. The model card publishes no evaluation results. There is no way to verify the capability claims at this time.
• License ambiguity. The base model ships under Apache 2.0. Training on Claude-generated outputs raises a separate licensing question. Anthropic’s usage policies restrict using Claude outputs to train competing models. The model card does not address this. Anyone deploying this in a commercial context should check that before proceeding.

Our take

The honest summary: this is a style-transfer fine-tune, not a capability transfer. You get a small, fast, offline model that formats responses in a Claude-like way. That is useful in the right context. It is not a replacement for a frontier model on complex reasoning tasks.

The 128K context in under 1GB is the genuinely interesting part. Most small local models cap context at 4K or 8K tokens. A model that can actually process a long document offline without a paid API has real applications in document review, local search, and privacy-first workflows.

The missing benchmarks are a problem. “Improves coding and instruction following” is not a number. Until someone runs evals, treat capability claims as anecdotal. If you are considering local AI models for a specific business task, talk to us before committing to a stack based on untested weights. We have written more about what community model releases actually deliver over on the Lumien news feed.

Run Q8_0 if your hardware supports it. Q4_K_M saves about 450MB but loses quantization precision. For a model already operating at 1B parameters, that precision loss matters more than it would on a 70B base.

Source: Marktechpost