Open Models Almanac
Methodspost-trainingpolicy distillation

single mention · post-training

Dual-Query Distillation

Dual-Query Distillation appends two learnable queries to the visual and textual tokens, one targeting the current observation and one the future observation.1 The query states are distilled from two complementary teachers: a depth teacher providing geometric supervision and a causal video teacher providing temporally grounded representations.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we append two learnable queries, , to the visual and textual tokens, where targets the current observation and targets the future observation at horizon — From Foundation to Application: Improving VLA Models in Practice
  2. distilled from two complementary teachers: a depth teacher, LingBot-Depth [28], which provides explicit geometric supervision, and a causal video teacher, DINO-Video, which provides temporally grounded visual representations — From Foundation to Application: Improving VLA Models in Practice

Also called dual-query distillation framework.

Filed under post-training › policy distillation