MORPHEUS: Skyfall AI’s Benchmark That Never Resets the World

Skyfall AI has released MORPHEUS, a persistent enterprise simulation platform designed to test continual reinforcement learning (CRL) agents in environments that never reset. Published in July 2026, the benchmark is built on the Big World Hypothesis (Javed and Sutton, 2024) and forces agents to adapt to structured, ongoing change rather than isolated episodes. It includes a six-metric evaluation protocol, two tested tasks, and open-source evaluation code, giving researchers a reproducible way to compare CRL algorithms on enterprise-style operational problems.

What happened

Detail	Fact
Release	MORPHEUS by Skyfall AI, July 2026
Theoretical basis	Big World Hypothesis, Javed and Sutton (2024)
Failure injection rates	Light 5%, Realistic 8%, Moderate 15%, Aggressive 30%
Failure types available	11 (including missing_data, dependency_failure, rate_limit)
Composite reward upper bound	0.50 per configuration (assumes zero failures, minimum cost, full throughput)
Policy initialisation models	Gemini 3.1 Pro (trajectory collection) + Qwen3-14B (SFT fine-tune)
Base RL optimiser	PPO for all baselines
Environments evaluated so far	2 of 5
Evaluation code	Open-source at Skyfall-Research/morpheus-evals

Standard reinforcement learning benchmarks reset the world at the end of every episode. Real business operations do not. A logistics network, a cloud resource scheduler, a support ticket system: these carry the consequences of past decisions forward indefinitely. MORPHEUS is built around that reality.

The platform enforces three properties across every environment. Persistence means past decisions compound into future dynamics. Non-stationarity means any fixed policy eventually degrades. Operational complexity means no fixed optimal policy exists at all. Each environment is a self-contained TypeScript plugin that exports Operational Descriptors (ODs), a simulation scheduler, seed data, and documentation.

How MORPHEUS creates non-stationarity

Two engines work together to prevent agents from finding a stable footing. The first is a failure injection engine that inserts typed disruptions between steps of an Operational Descriptor execution. It draws from 11 failure types and runs at one of four preset rates. The second is an asynchronous configuration shift controller that changes those failure presets and demand levels at fixed timestamps, deliberately out of sync with gradient updates. That last detail matters: if shifts aligned with training updates, agents could use update frequency as a clock to anticipate changes.

Reward comes from three operational verifiers logged natively by the platform: failure event signals, financial ledger status, and resource throughput. The composite reward weights them at w_f = 0.5, w_l = 0.25, and w_p = 0.25 by default. Under ideal conditions (zero failures, minimum cost, full throughput) the per-configuration upper bound is 0.50.

Policy setup and evaluation protocol

Because the action space is large, pure reinforcement learning from scratch is impractical. MORPHEUS uses a two-stage pipeline: Gemini 3.1 Pro collects trajectories using the ReAct framework, then those traces fine-tune Qwen3-14B via supervised fine-tuning (SFT). All algorithm baselines then start from this shared SFT checkpoint so that comparisons isolate continual learning behaviour rather than basic competence.

The research team evaluated four algorithm families across two tasks. Task 1 is dynamic resource allocation under structured drift. Task 2 is scheduling under drift with delayed effects.

Family	Mechanism	Task 1 result	Task 2 result
PPO	No CRL mechanism	Failure baseline	Baseline reward
HER	Hindsight replay	Mid reward	Best reward, top rank
EWC	Weight consolidation	Best reward	Weakest reward
LCM	Latent context model	Fastest adaptation	Best adaptation

No family dominates. EWC leads on reward in Task 1 but collapses in Task 2. LCM adapts fastest in Task 1 but loses that edge under delayed rewards in Task 2. PPO and HER generally adapt only in the first configuration and then fail to adapt in later regimes. Critically, mean performance gaps sit near 1.0 for every method. That signals a large settled-state deficit, not a minor tuning problem waiting for a hyperparameter fix.

Beyond cumulative reward, the platform uses six metrics: per-configuration reward, adaptation speed, forgetting, recovery time, stability, and performance gap. Adaptation speed is the headline metric, counting the steps until the running-average reward reaches half the upper bound.

Why it matters

Most published CRL results come from benchmarks where the world resets cleanly. That makes algorithms look more capable than they are when deployed in real systems. MORPHEUS makes the problem harder in a way that reflects actual enterprise environments: compounding history, externally triggered regime shifts, and delayed feedback loops.

The finding that no algorithm family wins consistently is genuinely useful. It tells practitioners that picking a CRL method based on a single benchmark number is unreliable. The near-1.0 performance gap across all methods also suggests the field has a long way to go before any of these approaches are ready for production enterprise use cases.

For teams building AI integrations into business workflows, this research reinforces a practical lesson: an agent trained on historical data or a static simulation will degrade in live environments. The degradation is not random, it is structured and predictable once you understand what causes regime shifts in your domain.

Our take

MORPHEUS is a well-scoped research contribution, not a product. The open-source evaluation code is the most immediately useful part. The two-stage initialisation pipeline (frontier model for trajectories, smaller model for SFT) is a sensible pattern that mirrors how production ML teams already work.

The weaknesses listed in the paper are honest. Only two of five environments are evaluated. The upper bound is optimistic by design. Shifts are externally triggered rather than emerging from compounding decisions, which is actually closer to some enterprise scenarios (scheduled maintenance windows, fiscal quarter resets) but not all of them. Reward weights are research variables and not validated against real operational KPIs.

If you are tracking the state of continual learning for enterprise AI, this is worth reading closely. If you are a practitioner trying to decide whether to deploy a RL-based agent in an operational system today, the near-1.0 performance gaps across every tested method are your answer: the benchmark exists precisely because the problem is unsolved. Approach self-improving agent architectures with that in mind.

What to do about it

1. Read the open-source evaluation code at Skyfall-Research/morpheus-evals before investing in a proprietary CRL evaluation setup.
2. If you are running RL-based agents in production, audit whether your internal benchmark ever resets state. If it does, your offline results may overstate real-world performance.
3. Use the six-metric protocol (especially adaptation speed and performance gap) as a template for your own evaluation, even outside the MORPHEUS environments.
4. Hold off on EWC as a default CRL choice: it wins on one task and fails on another, and Task 2’s delayed-reward structure is more representative of real business processes.

The near-1.0 performance gaps across all tested methods are the most important number in this paper: they confirm the problem is real and unsolved, not just theoretically interesting.

Source: Marktechpost