Open Dreamer: Full Training Recipe for a 1.6B Dreamer 4 World Model

A team called Reactor has published Open Dreamer, an open-source reproduction of the Dreamer 4 world-model pipeline built in JAX and Flax NNX. Two GitHub repositories cover the full training stack: a causal video tokenizer, a 1.6-billion-parameter action-conditioned latent dynamics model, rollout generation, and FVD scoring. A live browser demo streams a generated Minecraft world in real time, with a toggle that switches frame-by-frame between the real game and the world model. The training recipe, full configs, and hard-won stability fixes are all published.

What happened

Detail	Value
Team	Reactor
Framework	JAX / Flax NNX
Dynamics model size	1.6B parameters
Training steps	200,000
Peak learning rate	3e-4 (Muon optimizer, WSD schedule)
Model FLOPs utilization	57-58% (vs. 60% healthy benchmark)
Tokenizer compression	~100x
Video resolution	360×640 padded to 368×640
Latent tokens per frame	512 at bottleneck width 16
What is missing	Behaviour-cloning and RL agent loop

Reactor released two repositories. next-state/open-dreamer contains the training pipeline: a causal video tokenizer, an action-conditioned latent dynamics model, rollout generation, and FVD (Frechet Video Distance, a standard measure of video generation quality) scoring. reactor-team/open-dreamer is a minimal local harness that generates frames from an MP4 and a matching action file. A third artifact is the browser demo on Reactor’s own runtime, which streams a generated Minecraft world and exposes a Game / Dream toggle that hands the stream to the world model one frame at a time.

The team started on CoinRun, a procedurally generated 2D platformer that trains on a single GPU, then scaled the same pipeline to Minecraft gameplay video in the style of OpenAI’s VPT project. Their stated constraint: no methods outside the Dreamer 4 paper, to keep the search space narrow.

How the architecture works

Both the tokenizer and the dynamics model share one block-causal transformer backbone. That backbone alternates between two attention types: space layers move information among elements within a single frame, while causal time layers move information between frames.

The tokenizer is a Masked Autoencoder (a transformer that learns by reconstructing randomly hidden patches), not a VAE. The team reports roughly 100x compression and says the design needs no KL or adversarial loss. MAE masking probability tops out at 0.9, and LPIPS perceptual loss is applied at weight 0.2 on half of all timesteps.

The dynamics model handles next-frame prediction and is trained with a combination of diffusion forcing, flow matching, and shortcut models. It also predicts the next action. Rollout is folded into per-timestep blocks of (previous action, state, policy). A key design constraint: world-model tokens cannot read the agent token directly, so policy information can only affect future states through the next predicted action.

Dynamics model config at a glance

• 30 block-causal layers, d_model 1920, 30 attention heads, 3 KV heads (grouped-query attention)
• Every fourth layer is a time-attention layer
• 32 learned register tokens per timestep
• packing_factor: 2 packs neighbouring tokenizer latents into each dynamics spatial token
• Time attention uses a 192-step sliding window
• Shortcut/bootstrap samples activate at step 100,000 at a 0.25 batch fraction
• EMA decay of 0.999

Why the stability section matters most

The team is direct: stability work consumed the largest share of their time. Their central observation is that most stability failures are invisible in the loss curve. MSE improves smoothly while generation quality degrades. This is the kind of problem that wastes serious compute before anyone notices.

Six documented fixes:

1. Muon replaced LaProp. LaProp spiked randomly and increasingly often across two runs totalling roughly 400 B200 GPU hours.
2. EMA weights are mandatory for diffusion inference. Using the live weights instead breaks generation.
3. Mixed precision is boundary-sensitive. Parameters stay float32; BF16 covers most matmul activations and attention inputs; float32 is kept for normalization and the dynamics flow output head.
4. Loss weighting. They use x-prediction with a v-space loss, which produces a weighting term similar to Dreamer 4’s but with a squared denominator. They report a small but noticeable quality improvement.
5. Minibatch barycentric optimal transport between noise and latent sequences made rollout generation more stable.
6. mu-parametrization was tested and dropped, partly because Muon already keeps hyperparameters steadier across model sizes.

What the compute story looks like

The team reports 57-58% model FLOPs utilization against a 60% benchmark for healthy transformer training. On a B200 GPU, the crossover between bandwidth-bound and compute-bound workloads sits at 292 FLOP/byte. Feeding 256 frames per GPU pushes past that point.

Sharding went the other way from what they expected. At 1.6B parameters, the model state (parameters, gradients, optimizer state, EMA) fit in roughly 24 GiB on a single B200. Activations were the real memory cost. After trying data parallelism, FSDP, tensor parallelism, and sequence parallelism, they settled on plain data parallelism plus activation checkpointing. Data loading was solved by pre-tokenizing the whole dataset into .arrayrecord files and using Grain with a GPU-side prefetch buffer; ffmpeg decoding was too slow to keep the GPUs fed.

From the CoinRun phase, an iso-FLOPs sweep put compute-optimal scaling at roughly N proportional to C to the power of 0.56 and D proportional to C to the power of 0.44, where N is model size, D is data, and C is total compute.

What is not included

The repositories do not contain a behaviour-cloning or RL training loop. A full Dreamer 4 BC/RL agent loop is listed as a roadmap item. The CoinRun policy work is not released. FVD scores are not published in the post.

Why it matters

World models like Dreamer 4 are increasingly relevant for any team building AI agents that need to plan or simulate outcomes before acting. Having an open, reproducible pipeline with real training configs and honest failure notes is genuinely useful. Most open-source releases skip the stability section; this one leads with it.

For teams watching the broader AI infrastructure space, the compute analysis is worth reading alongside AMD’s Helios AI server commitments from OpenAI and Anthropic: the B200 roofline numbers here illustrate exactly why next-generation memory bandwidth matters for large model training.

Our take

Open Dreamer is the rare open-source release that is more useful for what it admits than for what it claims. The six stability fixes alone are worth reading if you are training any large generative model. The absence of FVD scores and RL loops means this is a foundation, not a finished system. Teams considering AI integration for products that involve video understanding or agent simulation should watch this project closely, but should not count on it being production-ready today.

The compute detail is also a useful reality check. Roughly 800 B200 GPU hours just to discover that one optimizer (LaProp) was randomly spiking is a reminder that stability is expensive to find and cheap to publish. Budget accordingly.

Source: Marktechpost