Introduction

Vision-language-action (VLA) policies used for controlling robots are typically deployed with synchronous execution. If you watch demo videos from papers like π₀, π_0.5, or OpenVLA, you'll notice that the robot's motion is a series of movements with short pauses in between. Those pauses are the robot running inference. While it's thinking, the robot either idles or blindly executes stale commands while the next plan is being computed. That's a problem, especially if anything in the scene is moving.

The obvious fix is to use asynchronous inference, where we start computing the next action chunk while the robot is still executing the current one. However, this creates a sneaky problem: by the time inference finishes, the observation that triggered it is stale.

Existing works have taken different approaches to fix this. RTC uses an inpainting-style procedure that blends the new action chunk with the old one (great for action continuity), but the policy is still looking at a stale camera frame. VLASH goes further and rolls the robot's joint state forward to the future timestep, but leaves the image stale. So the policy sees a future robot state paired with a past camera frame, a combination it never saw during training. To make this work, VLASH needs a special fine-tuning procedure with temporal offsets baked in; this is doable, but it adds cost and complexity.

F2F-AP takes yet another angle and predicts optical flow to paint the future view onto the current frame, but without action conditioning, so it can only estimate where objects drifted, not where the robot's embodiment actually went. WAM sidesteps the staleness problem entirely and removes the VLA from the loop. This sits at a different point in the design space: RTC, VLASH, and F2F-AP all preserve an existing pretrained VLA and adapt its inputs (or outputs) to handle asynchrony, whereas WAM replaces the policy altogether. The tradeoff is between leveraging a capable, already-trained VLA versus the generality of a zero-shot world-model policy that needs no task-specific policy at all.

Method	Fixes image staleness	Action-conditioned	No VLA retraining
RTC	✗	—	✓
VLASH	✗	—	✗
F2F-AP	✓	✗	✗
WAM	✓	✓	✗ (replaces VLA)
DreamActVLA (Ours)	✓	✓	✓

So we wondered: what if the missing piece isn't a smarter correction, a partial rollout, or a new policy, but simply giving the VLA the one thing it was trained to expect: a temporally-consistent [image, state] observation?

The real problem wasn't asynchrony: it's the partial roll-forward. If we move the state forward but leave the image behind, and you've created a mismatch the policy has never seen. But roll both forward to the same future instant, and the policy gets a perfectly consistent $(I_{t+\Delta}, s_{t+\Delta})$ pair exactly what it was trained on. No distribution shift. No re-training. The policy doesn't even know it's running asynchronously.

Key Insight: Temporal alignment is the missing piece. In theory if both the image and the state were rolled forward into their future selves, async inference would becomes indistinguishable from sync. Furthermore, no fine-tuning changes required.

We validated this in three steps: first with a privileged simulator peek (MuJoCo snapshot/restore) as a proof of concept, then with a learned latent world model that works without the simulator, and finally on a dynamic LIBERO variant where latency actually hurts. The result: 94.2% average success across all four LIBERO suites vs. 96.8% sync oracle — a gap of just 2.6 points, while the no-WM async baseline collapses to 67.3%.

Contributions

• Temporally aligned asynchronous inference: By querying the policy with a temporally aligned future $[\text{image, state}]$ observation, DreamActVLA runs asynchronously with no adjustment to the fine-tuning process. It simply corrects the observation input rather than the action output (RTC) or relying on offset-aware training (VLASH).
• Action-conditioned latent world model: We train a U-Net in the latent space of a frozen Cosmos VAE to predict future visual observations from stale frames and committed actions, removing the simulator dependency of the privileged approach without modifying the VLA policy.
• Dynamic task benchmark: We develop a LIBERO benchmark variant with wall-clock-tied object motion (instead of the standard static objects) and evaluate the performance of asynchronous methods in this setting.

---

How we Implemented our approach

Privileged Future Observations

Let $o_t = (I_t, s_t)$ denote the full observation at timestep $t$, where $I_t \in \mathbb{R}^{H \times W \times 3}$ is the rendered camera frame and $s_t \in \mathbb{R}^{8}$ is the proprioceptive state vector (end-effector position, axis-angle orientation, and gripper aperture).

Given a current action chunk $a = [a_t, \ldots, a_{t+N-1}]$ of length $N$, we trigger asynchronous inference $K$ steps before the chunk ends, at step $t + N - K$.

At the trigger point, rather than using the current (stale) observation, we obtain a future observation $o_{t+N}$ by performing a brief rollout within the simulator:

The next action chunk is then computed as $\pi(o_{t+N})$ and is ready at the chunk boundary $t+N$ when the robot needs it. Because both $I_{t+N}$ and $s_{t+N}$ are drawn from the same simulated future instant, the observation pair is temporally aligned

DreamActVLA: The Latent World Model

The privileged rollout trick only works in simulation. On a real robot there is no MuJoCo state to save and restore, so we need a learned model that can predict the future frame from the stale observation and the planned actions alone.

Rather than training a full pixel-space video diffusion model — which would be slow and expensive — we build on the frozen Cosmos VAE as a compact visual representation and train a lightweight latent dynamics predictor on top of it. The key insight is that future-frame prediction is much easier in latent space than in pixel space, and the Cosmos encoder already captures the visual structure we care about.

Concretely, the pipeline has four stages:

1. Encode: The frozen Cosmos encoder maps the stale camera frame $I_t \in \mathbb{R}^{H \times W \times 3}$ to a compact latent $z_t \in \mathbb{R}^{16 \times 28 \times 28}$.
2. Condition: A small trainable action encoder compresses the action chunk $A_t = [a_t, \ldots, a_{t+\Delta-1}]$ into a fixed-size conditioning vector that is broadcast into every U-Net layer via cross-attention.
3. Predict: A trainable U-Net (frozen down-blocks, trained up-blocks) processes $z_t$ conditioned on the action encoding and outputs a residual $\Delta z$. The predicted future latent is $\hat{z}_{t+\Delta} = z_t + \Delta z$.
4. Decode: The frozen Cosmos decoder maps $\hat{z}_{t+\Delta}$ back to pixel space, producing $\hat{o}_{t+\Delta}$ — the predicted future observation the VLA will act on.

During training, we supervise $\hat{z}_{t+\Delta}$ against the ground-truth future latent obtained by encoding the corresponding future frame from the LIBERO expert demonstration dataset. At inference no dataset or simulator access is needed: the model takes only the stale frame and the committed action chunk and produces the corrected observation in one forward pass.

Why residual prediction? In manipulation tasks most of the scene — the table, the background, lighting — does not change between $t$ and $t+\Delta$. Predicting only what changes ($\Delta z$) is a much simpler regression target than predicting the full future latent from scratch, and it keeps the model compact enough to run in real time alongside policy inference.

Dynamic Tasks

The privileged and world-model conditions above both use the standard (static) LIBERO suites, where objects sit still until the robot touches them. Latency costs time there but rarely changes what the right action is. We wanted to test the harder case, where the world keeps moving while the policy thinks, so we built a dynamic LIBERO variant in which the goal objects slide across the table at a constant speed. Now a policy acting on a stale frame is aiming at where the object used to be, not where it is, which is exactly the regime where catching up to the present should matter most.

Getting this benchmark to measure the right thing took a few iterations, and a few of the design choices turned out to matter a lot:

• Drift is tied to control steps, not wall-clock time. Objects move a fixed distance per control step (say 0.4 mm/step, about 0.8 cm/s), so how long our GPU happens to take has no effect on the physics. Every method sees the same world, and we model latency explicitly with a delay parameter $\delta$ (counted in 50 ms control steps). Our first version tied drift to wall-clock time, which quietly handicapped whichever condition ran slowest on the GPU. We only noticed by comparing per-episode timings.
• Only the goal objects drift. If every free object moves, the scene falls apart: items slide off shelves and pile up in the workspace. So we restrict drift to the objects named in the task's goal, which perturbs the task without destroying it.
• Objects freeze once they are placed. An object stops drifting as soon as its goal condition is met. Without this, multi-step tasks become impossible: the first item slides back out of the basket while the robot goes for the second, and even a perfect zero-latency policy scores 0%.
• We drop tasks where drift breaks the instruction. Goals defined by spatial relations (like "the bowl next to the ramekin") stop making sense once things move, so we leave them out. We also tune the drift speed so a zero-latency policy still lands around 50 to 75%, which keeps the benchmark hard enough to be interesting but not so hard that everything fails.

---

Does Our Approach Actually Work?

Does Alignment Help? Yes!

We evaluate on the LIBERO benchmark using the π₀-LIBERO checkpoint without any modification or fine-tuning. Each of the below three conditions is evaluated over 50 rollouts per task:

• Sync: standard synchronous baseline.
• Future Obs (Privileged): async with simulator oracle providing the aligned future observation.
• Future-State Only: async with only the state rolled forward — the VLASH-style mismatch to serve as the failure baseline.

Each policy call produces an action chunk of length $N = 15$, and the asynchronous conditions trigger the next inference $K = 10$ steps before the current chunk ends. All other parameters are identical across conditions, so any difference in outcome reflects the inference strategy alone.

The privileged condition achieves 92.5% success rate (only a 1% reduction relative to Sync) while reducing mean task completion time to 7.61 s, a 1.17× speedup. This confirms that temporal alignment alone is sufficient for near-oracle performance. Because the privileged access gives us a perfect future observation, these results establish the upper-bound oracle that a learned world model should aim to match.

Future-State-Only Ablation

We also tested a future-state-only condition, where we rolled forward just the state while keeping the current camera image. This is essentially what VLASH does at inference time, but applied to a stock π₀ checkpoint without any offset fine-tuning.

Here the Task Success Rate falls to 27.6%. The policy simply can't handle the mismatched input. This is exactly the failure mode that motivated DreamActVLA: if you're going to roll forward, roll forward everything. Align both modalities and the policy doesn't even notice it's running asynchronously.

DreamActVLA: Learning to Align in the Real World

The privileged results shows that temporal alignment is sufficient for near-oracle performance. But can our latent world model replicate this at scale (replacing the simulator peek with a learned prediction across all LIBERO suites.)

Before examining task success rates, we verify that the world model produces visually plausible predictions. The video below shows one episode step-by-step: the stale delayed observation the policy would normally see, the WM-predicted frame it actually receives, and the ground-truth current frame from the simulator (for both the primary and wrist cameras.)

We then evaluated the latent world model on all four LIBERO suites using π_0.5-libero via the OpenPI server: 50 rollouts per task, 10 tasks per suite, 500 episodes per suite. The async conditions use a fixed inference delay of 4 steps. Three conditions are compared: the official synchronous π_0.5 baseline, async inference with no world model (stale observations), and async inference with DreamActVLA's latent WM providing corrected observations.

Dynamic Tasks

These experiments use the π_0.5-LIBERO checkpoint (10-action chunks, 20 Hz control) with our learned latent world model, against a zero-latency oracle and async without WM (proprioceptive state is current as the roll-forward is free, but the images are $\delta$ steps stale and the full chunk executes late). Each cell is 100 episodes, so treat differences under ~5 points as noise.

As a scoping note: this no-WM condition degrades through timing misalignment and visual staleness. Our simulated control stack absorbs the jerkiness and chunk-splice discontinuities that can dominate asynchronous failures on real hardware (as VLASH observed on its SO-100 tasks). Hence, these numbers measure the staleness problem in isolation, not true real-robot instability, and should not be interpreted as a reproduction of real-world naive-async failures.

How fast can the world move?

Drift (mm/step)	Sync	Async (no WM)	DreamActVLA (WM)
0 (static)	90	67	85
0.4	71	40	69
0.7	49	23	40
1.0	46	14	22

First, scene motion is expensive for everyone, even the zero-latency oracle falls from 90% to 46%, because objects keep moving during open-loop chunk execution no matter how fresh the observation was. Second, the world model recovers most of the asynchronous gap at mild-to-moderate drift: at 0.4 mm/step it sits at 69% against the oracle's 71%, with the no-WM condition at 40% (asynchronous execution at essentially synchronous accuracy with a frozen policy.)

A notable limitation. At the most aggressive object velocity drifts (1.0 mm/step) the world model's gains shrink to 22%. This reason is quite straightforward: our predictor sees a single stale frame plus the robot's actions, and one frame carries no velocity information, so the model renders drifting objects right where the stale frame had them. It nails the robot's own motion, but a fast, independently moving object is essentially invisible to it. The fix is to give the model a sense of motion, for example by conditioning on two consecutive frames, which is one of the directions we discuss in the conclusion.

How much latency can it absorb?

We can note that Async without WM collapses monotonically as latency grows (58% → 5%); the world model holds 56–69% across the entire 100–400 ms range, and 320+ ms is the measured latency of inference-optimized 7B VLAs, so the right end of this axis is not unusual. The dips at 100 and 300 ms are an artifact worth owning: our predictor was trained with a fixed 4-step horizon, and those delays force zero-padded action windows; its best points sit exactly at the horizons it serves natively, so this curve is a lower bound, improvable with horizon-matched training.

What does the world model cost? On our hardware, a policy chunk takes $T_\pi = 226$ ms to generate (≈ 4.5 control steps, which is why our 200 ms condition is essentially this policy's real latency). The world model adds just $T_{wm} = 50$ ms on top, run once per chunk before the policy sees the corrected frame. Now in practice, that is a single extra control step of reaction latency, roughly a fifth of one policy inference. It barely grows with delay (two-hop prediction for longer horizons costs only 51.7 ms). To note, the policy still runs once per chunk and every action in the chunk is used, so we get that temporal alignment without sacrificing throughput.

Would a smoothing async method do better?

No, and the reason is quite revealing. We ran Real-Time Chunking on a π_0.5 checkpoint fine-tuned for asynchronous execution (50-action chunks) on the same drifting benchmark. It retains 51.5% on LIBERO-Object (from ~99% static) and 40% on LIBERO-10 (from ~93%), retention ratios of 0.52 and 0.43. Those are statistically indistinguishable from our zero-latency oracle's retention (0.54 and 0.40). In other words, a stronger policy with longer chunks, async fine-tuning, and action inpainting loses exactly as much to scene motion as a method with no latency at all. That tells us scene motion is not really a latency problem: it is a separate axis that none of today's async techniques touch, ours included. Closing it is what the velocity-aware extension below is for.

Discussion and Conclusion

We have shown that temporal alignment, meaning the policy conditions on a consistent estimate of the present, is enough to recover most of the accuracy lost to asynchronous inference, with no changes to the VLA or its fine-tuning. A privileged simulator oracle first confirmed the hypothesis (92.5% vs 93.5% sync), and DreamActVLA then realized it with a learned latent world model, recovering +26.8pp over the async no-WM baseline to reach an average success of 94.2% across all four LIBERO suites, within 2.7pp of the synchronous inference variant. Once again, it is also worth being clear about where DreamActVLA sits relative to prior work: RTC corrects the action output, and VLASH rolls the state forward with offset-aware fine-tuning, while DreamActVLA corrects the observation input and leaves the policy frozen. Because it acts on a different part of the pipeline, it could in principle be combined with either.

Future Work

• A velocity-aware world model. The biggest gap at the moment is the single-frame predictor cannot see velocity, so moving objects can slip past it. Conditioning the world model on two consecutive frames, or on auxiliary motion signals like optical flow or object trajectories, would give it a sense of motion, letting it predict where things are heading rather than only where they were. We expect this to be the single largest lever on the dynamic benchmark, since it targets the one failure mode (scene motion) that no current async method, including ours, addresses.
• Matching the prediction horizon to the delay. Our world model was trained at a fixed 4-step horizon, which is why its accuracy dips at delays it has to pad or chain to reach. Training across a wide range of horizons should smooth this out and likely raise the numbers we report here, as those curves are effectively a lower bound.
• Real-robot deployment. Everything here runs in simulation, where we can fall back on a privileged oracle to sanity-check predictions. But on hardware there is no such fallback, so the next step is to validate that the world model's frames are good enough to drive a real arm under genuine inference latency.

---

Project Code

The implementation code for this project is split across the following repositories:

• Privileged Future Observations: github.com/themboutidem/future-obs-pi0
• DreamActVLA: github.com/Somdit/DreamActVLA
• Dynamic Tasks: github.com/SpaceQuark/DreamActVLA

---

Citation

If you find this work useful, please cite it as:

@misc{dreamactvla2026,
  title  = {Temporally Aligned Asynchronous Inference for Vision-Language-Action Models},
  author = {Zhang, Chi and Mkpong, Mboutidem and Chen, Richard},
  note   = {Equal contribution},
  year   = {2026},
  url    = {https://dream-act-vla.github.io}
}