VSpark: Speculative Decoding for Autoregressive Vision-Language-Action Models

Suho Yoo1*   Youngjoon Jang2*   Hyebin Cho1   Taeyeop Lee1   Jaewook Jeong1   Joon Son Chung1

1KAIST2VGG, University of Oxford*Equal contribution

suho.yoo@kaist.ac.kr

00

Exact verification, faster decoding

Left: speedup bars on LIBERO for π0-FAST and ActionCodec, with VSpark highest. Right: a Franka arm opening a drawer, with the rejected-token branch outlined in red 23.6 mm away from the corrected branch, an error-over-time plot, and token rows showing the continuation a3 to a7 reused after correcting a2 to A2.
Accelerating VLAs with VSpark. (a) VSpark accelerates autoregressive vision-language-action models by up to 2.10× on LIBERO. (b) Different action token sequences can represent similar robot motions, and predictions along a rejected branch can still match those under the corrected prefix even after a token mismatch. While conventional speculative decoding discards these predictions, VSpark exploits this retained predictive information to improve drafting while retaining exact target verification.

Can speculative decoding be made effective for VLAs with compressed action tokenizers while keeping verification exact?

Autoregressive VLAs pay one large-model forward pass per action token. Speculative decoding drafts several tokens and verifies them in one pass, so decoding remains lossless. For robot actions, near-correct drafts are still rejected, which limits the speedup.

01

Decoding demo: AR, DSpark and VSpark

A 22-token ActionCodec chunk on LIBERO-Goal, replayed from real decoding traces. One timestep is one target forward pass. After a rejection, DSpark discards the rest of the drafted path, while VSpark saves the target's predictions along it and proposes them in the next pass. In this chunk, all three output the same tokens. Both DSpark and VSpark use tree decoding; only the path through the accepted tokens of each draft tree is shown.

Timestep 0 / 22
accepted rejected target token discarded continuation reuse
02

Three design choices on the draft side

VSpark builds on DSpark. It changes how the draft is trained and which candidates are verified, and leaves the target, tokenizer and verification rule unchanged.

Overview diagram. Training: (a) rollouts with action noise between simulator and target model, (b) feature alignment with an L1 loss on target features that condition the draft model. Inference: the target model verifies draft tokens with strict verification, and (c) continuation reuse carries the target continuation from a rejected branch into the next cycle's tree.
Overview of VSpark. During training, (a) rollouts with calibrated action noise collect target supervision at perturbed states, and (b) feature alignment uses target representations to condition and supervise the draft. During inference, (c) continuation reuse brings the target's continuations along rejected branches into the next draft tree. Strict verification checks all candidates under their current prefixes, preserving lossless decoding.
(a) Training data

Rollouts with calibrated action noise

Roll out the target with action noise and label every visited state with its clean prediction. The noise matches the draft's own action errors.

(b) Training objective

Feature alignment

Condition the draft on the target's hidden features and regress them alongside cross-entropy. τ on π0-FAST rises from 3.22 to 3.87.

(c) Inference

Continuation reuse

Reuse the target's predictions along rejected branches as candidates in the next draft tree. No training and no extra target compute.

Two draft trees. In cycle t the token a2,1 is rejected, but the target continuation a3,1, a4,1 below it is accepted under that branch. In the next cycle, rooted at the correction a2, that continuation is inserted as a branch.
Continuation reuse. During verification in cycle t, the target also predicts along rejected branches. VSpark reuses them to build the tree for cycle t+1, where they are re-verified.

Promoting continuations

In cycle t, the target accepts a0,1 a1,1 but rejects both a'2,1 and a'2,2, so it appends its correction A2,1. The same pass also verifies the tokens below the rejected ones. Under a'2,1, a'3,1 a'4,1 match the target and a'5,1 does not, so the target gives A5,1. Under a'2,2, a'3,2 matches and a'4,2, a'4,3 do not, so it gives A4,1. This yields two target continuations, a'3,1 a'4,1 A5,1 and a'3,2 A4,1.

In cycle t+1 the tree starts from A2,1. Both continuations are inserted below it with the top draft score under their parent, next to the draft's own candidates b':

s̃i(v) = maxu si(u)  if v ∈ Hi,  otherwise si(v)

The target re-verifies every inserted token under A2,1, so decoding remains lossless.

03

Fastest on every model and benchmark

Acceptance length τ and speedup over autoregressive decoding. All methods share the frozen target and exact tree verification; only the draft differs. Single RTX 4090.

Average speedup

Dashed line: autoregressive decoding (1.0×).

Per-suite results

04

BibTeX

@article{yoo2026vspark,
  title     = {VSpark: Speculative Decoding for Autoregressive
               Vision-Language-Action Models},
  author    = {Yoo, Suho and Jang, Youngjoon and Cho, Hyebin and
               Lee, Taeyeop and Jeong, Jaewook and Chung, Joon Son},
  journal   = {arXiv},
  year      = {2026}
}