Open Models Almanac
ModelsLingBot-VLA 2.0

Robbyant (Ant Group)

LingBot-VLA 2.0

What LingBot-VLA 2.0’s own documents say it is built from — every method with the sentence that describes it.

Parameters
6.4B
Attention
6B; ~20K h dual-arm data across 9 embodiments (v1)
Input
text+image -> action
Licence
Apache-2.0
Released
2026-07-07

Against the consensus recipe

Its documents state 1 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

filed at the root 1

Mixture-of-Transformers (MoT) core

“These components are organized via a Mixture-of-Transformers (MoT) architecture”

token mixer 2

softmax attention 2

Blockwise causal attention core

“we implement blockwise causal attention for modeling the joint sequence”

Shared self-attention across modalities core

“vision-language and action modalities are processed through distinct transformer pathways, coupled by a shared self-attention mechanism”

channel mixer 8

mixture of experts 8

Mixture of experts core

“we introduce a token-level loss-free MoE architecture for large-scale multi-embodiment VLA pretraining.”

expert routing 1

Sigmoid-based expert routing core

“To avoid the strong competition among experts induced by softmax normalization, we apply a sigmoid-activated function to the router logits”

expert load balancing 3

Auxiliary-loss-free load balancing core

“we adopt an auxiliary-loss-free strategy inspired by DeepSeek-V3 [19] to promote load balancing within the MoE architecture.”

Per-expert routing correction bias core

“each expert maintains a routing correction bias.”

Sign-based expert-load correction-bias updates core

“At each load-balancing update, the correction bias is adjusted according to the sign of the deviation from the mean expert load:”

shared experts 1

Shared expert isolation core

“Fine-grained expert segmentation and shared expert isolation allow universal priors and specialized embodiment/task patterns to coexist…”

fine-grained experts 2

Fine-grained expert segmentation core

“Fine-grained expert segmentation and shared expert isolation allow universal priors and specialized embodiment/task patterns to coexist…”

Fine-grained routed experts with a shared expert core

“we adopt fine-grained expert segmentation and shared expert isolation, in which a lightweight shared expert preserves universal priors…”

positional encoding 2

Shared spatial and modality-specific positional embeddings core

“a shared learnable 2D spatial positional embedding for both RGB and depth tokens; and (2) a modality embedding that distinguishes the…”

UV positional encodings core

“At each scale, UV positional encodings—derived from a circular mapping of image coordinates—are injected to preserve spatial layout and…”

prediction head 1

Diffusion-policy action head used

“we first convert the depth into a point cloud and then predict an dexterous hand pose using a diffusion policy”

multimodal architecture 7

ConvStack core

“we adopt a convolutional decoder architecture, termed ConvStack, which is more appropriate for dense geometric reconstruction”

Hierarchical convolutional decoder core

“The decoder follows a pyramid structure composed of a shared convolutional neck and multiple task-specific heads.”

Learnable image queries core

“we apply the learnable queries corresponding to three-view operational images”

Perceptual-query token appending core

“LingBot-VLA-V2 appends current and future perceptual queries to the visual/text tokens.”

Separate RGB and depth patch embeddings core

“We apply separate patch embedding layers to the two input modalities: the 3-channel RGB image and the single-channel depth map.”

DINOv2 and Point Transformer grasp-policy conditioning used

“The policy is conditioned on RGB features extracted by DINOv2 (ViT-L/14) and point cloud features from a Point Transformer”

Depth injection optional

“depth injection”

filed at the root 4

Predictive dynamics modeling core

“Predictive dynamics modeling: future prediction is used as a proxy task”

Vision distillation core

“we adopt a vision distillation approach inspired by recent works”

Valid-pixel-masked L1 depth loss used

“The predicted depth map is supervised using an L1 loss applied directly to the ground-truth depth map. Only pixels with valid depth values…”

Video-adapted DINO and iBOT self-distillation used

“We train DINO-Video on 5M video clips spanning Internet, egocentric, and robotic data, using video-adapted DINO and iBOT self-distillation…”

distillation objective 4

Depth-representation distillation for geometric supervision core

“the current and future queries, , are trained to predict the corresponding depth representations from the current and future manipulation…”

Distillation-loss alignment of VLM queries with depth tokens core

“We align the VLM learnable queries and LingBot-Depth tokens by minimizing the distillation loss”

Distilling current and future perceptual queries from pretrained perception models used

“These queries are distilled from LingBot-Depth and DINO-Video, encouraging causal inference to capture both current scene geometry and…”

Depth distillation optional

“depth-free version and a depth-distilled version.”

auxiliary loss 1

Natural masks for missing depth regions core

“interpretation of missing regions (“holes”) in raw depth maps as “natural masks””

optimizer 1

Differential learning rates used

“we employ a differential learning rate strategy: parameters in the encoder backbone are optimized with a base learning rate of , while all…”

learning-rate schedule 1

Linear warmup followed by step decay used

“After warm-up, a step decay scheduler reduces both learning rates by a factor of 0.5 every 25,000 iterations.”

training precision 2

BF16 mixed-precision training used

“mixed-precision training using BF16 is enabled throughout to improve computational efficiency and reduce memory consumption”

Mixed-precision policy with BF16 storage and communication and FP32 reductions used

“performing reductions in torch.float32 to ensure numerical stability, while utilizing torch.bfloat16 for storage and communication”

training stability 1

Global-norm gradient clipping used

“Gradient clipping with a maximum norm of 1.0 is applied to stabilize optimization”

training parallelism 2

Fully Sharded Data Parallel used

“We employ Fully Sharded Data Parallel (FSDP)—a highly efficient PyTorch implementation of the Zero Redundancy Optimizer (ZeRO)—to shard…”

Action-expert-specific sharding groups used

“we construct specific “shard groups” exclusively for the action expert modules”

filed at the root 1

Robot data feature mapping core

“Define feature mapping (states / actions / images) from raw keys to unified feature space”

data sourcing 2

Real-world robot-data pre-training core

“20,000 hours of real-world data from 9 popular dual-arm robot configurations.”

Large-scale multi-source pre-training data curation used

“a redesigned data pipeline curates around 60,000 hours of pre-training data, including 50,000 hours of robot trajectories across 20 robot…”

data filtering 14

Dataset normalization statistics computation core

“Calculate normalization statistics over your dataset”

Human verification of robot state and video alignment core

“Human annotators are employed to identify discrepancies between the projected robot and the videos”

Masking fully missing depth patches core

“a depth patch (token) whose values are entirely missing is always masked”

Minimum valid-frame coverage filtering core

“using a 20% valid-frame ratio as the minimum threshold.”

Near-static episode filtering core

“If this duration accounts for more than 95% of the entire episode, the episode is also discarded.”

Probabilistic masking of partially valid depth patches core

“For patches containing a mixture of valid and invalid depth values, we assign a higher probability (e.g., 0.75 in our training) of being…”

Random sampling of fully valid depth tokens core

“we randomly sample additional fully valid depth tokens to complete the masking set”

Robotic trajectory quality filtering pipeline core

“we employ a redesigned data processing pipeline to filter noisy samples, yielding 50,000 hours of high-quality robotic data.”

Trajectory smoothness filtering using jerk and derivative Z-scores core

“An episode is discarded if either the jerk or any derivative Z-score exceeds a predefined threshold.”

Trajectory-level quality control core

“We then apply trajectory-level quality control to the reconstructed and standardized data.”

Vision-language-model filtering of egocentric videos core

“We first apply a unified video-level VLM pre-filter to all candidate videos, regardless of source.”

Automated anomaly screening used

“an algorithmic screening procedure automatically excludes episodes exhibiting technical anomalies”

Human review of filtered demonstration episodes used

“human reviewers validate the filtered dataset using synchronized multi-view video streams”

Left-right consistency checking for stereo pseudo-depth used

“we do the left-right check and filter out the inconsistent pixel values from the depth maps”

synthetic data 6

Automated vision-language-model annotation of manipulation videos core

“We generate these annotations with a fully automated pipeline built on a vision–language model”

VLM-based temporal subtask segmentation and instruction generation core

“we employ Qwen3.6-27B [25] to segment each manipulation video into a sequence of temporally contiguous subtasks and to generate the…”

Gaussian-noise augmentation of depth maps used

“we artificially corrupt depth maps with Gaussian noise and apply our masking strategy during training”

Generate pseudo-depth labels from stereo infrared pairs used

“we follow FoundationStereo [33] to compute the stereo disparity of the left-right IR pairs to obtain the pseudo depth labels”

Simulate real active RGB-D camera imaging to generate imperfect depth used

“we explicitly simulate the imaging process of real-world active RGB-D cameras to generate realistic depth observations with natural…”

Synthetic RGB-D data generation using self-hosted 3D assets used

“Our data curation pipeline consists of two parallel streams: a synthetic pipeline built on self-hosted 3D assets”

policy distillation 1

Dual-Query Distillation core

“we adopt a dual-query distillation framework inspired by recent works [35], where LingBot-Depth and our robotics-aware DINO-Video serve as…”

inference kernel 2

FlexAttention used

“we leverage FlexAttention to optimize computation”

Operator fusion using torch.compile used

“we apply operator fusion (via torch.compile) to reduce kernel launch overhead”

filed at the root 1

Open-loop evaluation optional

“Support open-loop evaluation.”

human & real-world evaluation 2

Randomized-order sequential model evaluation used

“All models are tested sequentially on the same hardware-task pair in randomized order”

Repeated trials for statistical robustness used

“Each model undergoes 15 trials per task-robot pair for statistical robustness”

1 more mentioned or evaluated
Additive Gaussian and shot-noise perturbations evaluated

“we corrupt the remaining depth values with additive Gaussian noise and shot-noise-like perturbations inspired by Kinect noise models”

filed at the root 4

55-dimensional canonical state-and-action representation core

“we use a 55-dimensional canonical vector representation for both states and actions.”

Unified representation for multi-part robot action signals core

“the unified representation supports arms, end-effectors, grippers, dexterous hands, waist, head, and mobile-base signals instead of only…”

Bundle adjustment on tracked visual-odometry points used

“mainly resort to the Bundle Adjustment (BA) on the tracked VO points to track the camera motion”

Semi-global matching used

“The rendered stereo images are then processed using the widely adopted semi-global matching (SGM) algorithm to produce sensor-like depth…”

unfiled 20

Causal video-representation distillation for temporal supervision core

“the future query is trained to predict the feature of the future frame at horizon .”

Condition depth reconstruction on the complete RGB image core

“our architecture provides the full, unmasked RGB image as a condition”

Conditional flow matching for action generation core

“the training objective is to characterize the conditional distribution through conditional flow matching”

Control the monocular-depth and depth-completion tasks through the depth-token masking strategy core

“The transition between these tasks is governed simply by the masking strategy within the Transformer.”

Masked depth modeling for RGB-D depth completion core

“we introduce Masked Depth Modeling (MDM) to achieve dense, pixel-aligned scene geometry”

Egocentric SLAM for camera pose estimation used

“we run egocentric SLAM to estimate camera intrinsics and per-frame camera extrinsics”

GPU memory optimization during training used

“Baidu Cloud's optimization solutions notably reduced our GPU memory consumption by 29.2% during model training.”

Hand pose estimation for MANO trajectory recovery used

“apply hand pose estimation to recover MANO parameters in the camera coordinate frame.”

Image augmentation with crops, flips, and synthetic degradations used

“Data augmentation includes random resized cropping and horizontal flipping, along with a set of synthetic image degradations—specifically…”

Initialize framewise camera extrinsics to the SE(3) identity used

“we initialize the framewise extrinsics using SE-3 identity”

Modular multi-camera RGB-D capture system used

“we build a scalable RGB-D capture prototype”

Online RGB-D tracking with SpatialTrackerV2 used

“we use the online version of SpatialTrackerV2 [34] that does not depend on the VGGT [27]-Frontend for initial pose and depth estimation as…”

Randomize object poses during demonstration collection used

“object poses (positions and orientations) are randomized within the workspace for each trajectory to prevent overfitting to specific…”

Retarget human hand-object interactions using 3D keypoint correspondence used

“human hand-object interactions are retargeted to the dexterous hand configuration via 3D keypoint correspondence”

Torch Compile inference optimization optional

“Enable Torch Compile for inference.”

4 more mentioned or evaluated
Block-wise depth masking for depth-completion evaluation evaluated

“We generate incomplete depth maps by randomly masking out spatial regions of varying sizes (blocks) from the ground-truth depth”

L2 action-regression loss evaluated

“For the loss function, L2 achieves the best average performance, improving over L1”

Mean-standard-deviation action normalization evaluated

“By preserving these larger corrective motions, MeanStd achieves the best average success rate of .”

Relative joint-action targets evaluated

“relative joint actions substantially outperform absolute joint actions”

Not used 1

Omit global bundle adjustment for efficiency not used

“There is no global BA applied to enable the efficiency.”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
huggingface.co/robbyant/lingbot-vla-v2-6bmodel cardRobbyantread
arxiv.org/abs/2607.06403technical reportRobbyantread
arxiv.org/abs/2601.18692technical reportRobbyantread
arxiv.org/abs/2601.17895technical reportRobbyantread
github.com/Robbyant/lingbot-vlacode repoRobbyantread
↑