Highlight-guided Attention Steering

Don't drop the video.
Scroll through the method.

A scroll-story companion for HAS. The same video stays on screen while scrolling transforms it from continuous evidence into a temporal prior, visual-token steering vector, and finally a gated attention intervention.

Rui Chu · Yingjie LaoTufts UniversityFrozen backboneECCV 2026
scroll controls the animation · reverse scroll reverses the method
Full video evidence
Watch the entire video.The same temporal object stays on screen while the method transforms around it.
prompt-conditioned temporal prior
V = log(Repeat(h,P) + ε)
Original logitsA
Steered logitsà = A + gβV
Generated summary

The model keeps the full temporal evidence, while allocating more attention to salient interactions and retaining lower-scored context for coherence.

00 / Start

One continuous video.

Scrolling is the animation timeline. No click is required to understand the method.

01 / Background

Hard selection deletes evidence.

Importance becomes a binary keep/drop decision. Bridging context can disappear before the MLLM ever sees it.

02 / Rewind

HAS restores every frame.

Low highlight means less emphasis, not deletion. The full stream remains available.

03 / Highlight prior

Scores become a temporal prior.

H(F,q) → Interp → MinMaxNorm. The curve aligns with the entire video timeline.

04 / Token microscope

A frame opens into visual tokens.

The scalar hₜ is copied over the P visual tokens that originate from that frame.

05 / Steering vector

Move the prior into log-space.

V = log(Repeat(h,P)+ε) creates an additive bias aligned to visual-token logits.

06 / Attention steering

Inject gβV before softmax.

Selected cross-attention heads redistribute mass while the backbone MLLM stays frozen.

07 / Output

Summarize with evidence intact.

The model generates with soft temporal guidance rather than a hard frame bottleneck.

Explore after the story

Scroll teaches. Controls let you perturb the mechanism.

The V2 page keeps a compact lab for the two most intuitive controls. The full V1 dashboard remains available separately.

Mechanism lab
Change hard-selection budget and steering strength.
5 / 12
0.95
Hard selectionevidence removed
Attention before → aftersoft bias
Ã(ℓ,m)i,: = A(ℓ,m)i,: + gℓ,m βℓ,m V,   with   V = log(Repeat(h,P)+ε).