Teams from General Intuition, Kyutai, and Epic Games introduced MIRA — a world model that fully simulates the Rocket League game environment for four players at once and draws each of them their own picture in real time. The key point: there’s no game engine inside at all. No physics engine that computes collisions between the ball and cars, no renderer that draws the arena. All of it is replaced by a single neural network that, from the player’s button presses, draws the next frame itself. In effect, the whole game “lives” inside the model’s weights, and you can run it locally without installing the original Rocket League. The authors released the training and inference code on Github and a dataset of ~10,000 hours of gameplay on Hugging Face. You can play right in your browser in the demo version.
A world model is usually a neural network that learns to predict how a scene will change in response to the player’s actions, and thereby works as a controllable simulator. Previously such models could obey a single player and treated everyone else as just part of the environment. MIRA obeys four at once and learns to understand whose action led to which change on screen.
Why a multiplayer world model is needed at all
The idea of a world model is for the network to learn the dynamics of the environment directly from experience: it builds an internal latent representation of the world and predicts future states conditioned on actions. Inside such a learned simulator you can train agents, evaluate them, and imagine the consequences of actions before committing to them in the real environment. But most existing models see the world through the eyes of a single agent. This makes it hard to apply them where explicit control over several participants matters: self-play reinforcement learning, multi-agent training, counterfactual policy evaluation.
The authors chose Rocket League because it’s a convenient testbed. It’s a game with fast physics, where cars and the ball collide at speed, and players simultaneously compete and cooperate. At the same time, the game makes it easy to collect data at huge scale and to log actions precisely. The difficulty is that moving from one player to four is not just adding inputs: the model must learn to attribute changes in the scene to the correct player, represent interactions between them, and stay coherent under any combination of actions. New ways to break appear that weren’t there before: the model can misassign who affects what, ignore the actions of one of the players, or entangle the effects of several.
How it works inside
MIRA works not with pixels directly, but in the latent space of a special video codec. Predicting in pixels is expensive: the model spends capacity on high-frequency texture detail that has nothing to do with dynamics, and it’s too slow for real time. So frames are first compressed by the codec into a compact latent representation, the world model predicts the next latent frame, and a decoder unfolds it back into video.
The codec itself is a video autoencoder built on a frozen DINOv3-L feature extractor. The key finding: if you build the latent space on top of a pretrained feature extractor rather than learning it from scratch, then predicting in that space becomes noticeably easier and more stable over long horizons. The feature extractor stays frozen; only a linear “bottleneck” and the decoder are trained.
The world model itself is a diffusion transformer with 5 billion parameters and factorized space-time attention. It’s trained with flow matching. An important trick here is diffusion forcing: each frame in the training sequence gets its own noise level, so the model trains on partially corrupted context. This emulates the situation at inference, where the model relies on its own imperfect predictions, and it keeps small errors from accumulating and dragging the picture off course over a long horizon.
To keep the four views consistent, the four players’ pictures are stacked into one grid and predicted together. Spatial attention then spans all the views at once, and the model draws the shared world, the ball, and the other cars the same way from all four viewpoints in a single pass. Each player’s actions are encoded separately and fed into the model through adaptive layer normalization. There’s also a trick called action dropout: during training, a random player’s actions are sometimes replaced with a special “action absent” token, and the model learns to predict that player from the scene alone. At inference this lets the model itself play for any player the user isn’t controlling.
How it was trained and on what data
The dataset consists of 10,000 hours of recorded Rocket League matches in 2v2 mode, generated entirely by bots playing against themselves. There is no human data in the training set. Each match yields four synchronized first-person recordings, one per player. Each recording is video plus the player’s action stream plus the game’s physics state. Every car is driven by Nexto, the strongest publicly available bot for this game, trained via self-play reinforcement learning.
The game’s physics state (positions, velocities, orientations of the ball and cars) is logged separately at 120 Hz. The world model doesn’t see it and learns from pixels and actions alone. This state is needed as reference ground truth for evaluation: from it the authors check whether the model encodes physically meaningful quantities in its features.
What emergent properties appeared
The model was trained only to predict the next frame from past frames and actions, but its behavior showed patterns it wasn’t directly taught.
First: the model seemingly models a “theory of mind” for players whose actions are withheld. Thanks to action dropout, at inference an uncontrolled car keeps playing plausibly, moving and contesting the ball like a real player, with no action supervision at all. And the bots that generated the training clips had access to the full game state, whereas the model recovers their complex decisions from pixels alone.
Second: the model keeps all four views mutually consistent. A car that has left one screen stays in the others, and when it returns to frame it appears where the others place it. The same holds for events: when the ball goes into the net, the “SCORED!” callout and the goal explosion appear simultaneously across all four screens.
Third: the model generalizes beyond the training action distribution. For instance, scenes where all cars stand still never appear in the data (recorded play is always in motion), yet the model still holds such a scene stable, and cars don’t drift apart on their own. Live humans play quite unlike bots: bots press buttons fast and uniformly (around 390 actions per minute), while people are slower and far more varied. Even so, in the live demo under human control the model stays responsive and coherent.
Fourth: desynchronized views recover on their own. Occasionally one view gradually diverges from the others and degrades into noise: usually when a player enters a regime the bots never produced, for example standing still in the goal. The model then pulls it back, within moments, to a state consistent with the others, though it was never specifically trained for this.
Benchmark results
Since the exact future in such a game is unknowable, judging a rollout by pixel-by-pixel match to a reference is meaningless. The authors devised a set of metrics that probe physics and controllability, not just appearance. The main distributional metrics are gFID, gFVD, and gFDD (variants of Fréchet distances between generated and real video, lower is better). Separately there’s the Action Recoverability Ratio (ARR): a probe tries to determine from the generated video which buttons were pressed, and the metric shows how faithfully the model renders the actions it was given. ARR = 1 means actions are as recoverable from the generation as from a faithful reconstruction. There’s also a game-state probe that reads the positions and velocities of the ball and cars from the model’s internal activations and checks them against the true physics. The authors validated both metrics against human judgment: ARR matches human perception of controllability with a Pearson correlation of 0.84 and Spearman of 0.93.
The latent space is essential, and the gap here is huge. A pixel-space model without the codec trails by an order of magnitude: gFID 104.9 versus 10.7 for the latent model, gFVD 1456.3 versus 163.1, and on controllability ARR 0.61 versus 0.91. Even with the JiT recipe the pixel-space model stays worse, and its rollout drifts into warped texture within a second.
The feature extractor must be pretrained. An extractor trained from scratch renders sharper (PSNR 32.2 versus 29.7), but its latent space is much harder to generate: gFID jumps to 22.5 versus 10.7, and over a long rollout it drifts 1.7x more than the frozen DINOv3-L. The variant that distills DINO into a from-scratch extractor sits in between (gFID 15.7, drift 1.3x). The authors attribute this to the smoothness of pretrained features: nearby states map to nearby latents, so a wrong prediction stays plausible and is absorbed.
Diffusion forcing clearly beats teacher forcing. At the 4-second horizon, diffusion forcing gives gFID 10.7 versus 32.5, gFVD 163.1 versus 944.1. And beyond the training window teacher forcing degrades outright and holds gFID roughly 10x higher over a five-minute rollout (Table 8 and Figure 9 on p. 23 of the paper).
Few-step distillation (PSD) also helps: it beats the baseline at every number of sampling steps and stays stable even at a very small count, which is what makes real time possible.
The scaling picture is as expected. Models were grown from 100 million to 5 billion parameters, and quality improves monotonically: larger models converge faster and to lower error. The game-state probe confirms this from the physics side: ball-position readout error drops from 2130 for the 100M model to 1448 for the 5B model.
The steepest gains are between the smaller scales, after which returns diminish: 2.5B and 5B reach comparable quality at this data budget. Separately interesting is the multiplayer regime: training the model on four players from scratch at a small budget is bad (the model collapses); it’s better to first pretrain on a single player and then continue training on four, starting from the single-player checkpoint. At double the data budget the collapse disappears, and a mix weighted toward multiplayer (75% multiplayer, 25% single-player) gives the best gFID of 9.4.
Where the model still errs
The model is stable over long rollouts, so the remaining failures are rare and localized. Most of them stem from two causes: a context window too short to hold state across a whole match, and imbalances in the data, where rare events are undertrained and a few common ones overtrained.
The ball, left unattended, tends to roll toward a goal on its own: a still ball is rare in the data, and the model’s learned prior prefers continued motion. The player’s car occasionally jumps or boosts without a button press, especially at the kickoff from center field, where the bots almost always start the same way. The clock and score drift over time, usually at transitions (a goal, crossing a time milestone), because such moments happen once per game and are thinly represented in the data. In the single-player version there’s a separate problem: cars can “merge” into the ball, but in the multiplayer model, where it sees all four cars, this no longer happens.
Bottom line
MIRA is the first interactive multiplayer world model for an environment this dynamic with complex physics. It’s controlled by four players at once, stays coherent over hour-long rollouts, and runs in real time at 20 frames per second on a single Nvidia B200. Three decisions carried the result: predicting in a latent space on top of a frozen pretrained feature extractor (this gave nothing in reconstruction, but it’s precisely what stopped drift over long rollouts), diffusion forcing with few-step distillation for stability and speed, and conditioning on all four action streams at once. The authors state the limitations honestly: a single game, one type of bot, short memory, and data imbalances. All of it, together with the dataset, code, and a live demo, is released openly to push world model research forward.
