NVIDIA
GR00T N1.7
What GR00T N1.7’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- 3.1B
- Attention
- Cosmos-Reason2-2B VLM (System 2) + flow-matching DiT action head (System 1)
- Input
- text+image -> action
- Licence
- NVIDIA Open Model License (commercial); GitHub says Apache-2.0: check
- Released
- 2026-04-17
Against the consensus recipe
Its documents state 0 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertsnot in its documents
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modenot in its documents
- Multi-Token Predictionnot in its documents
- FP8not in its documents
filed at the root 3
“A 32-layer DiT takes the VLM's output and live robot state, then denoises them into precise motor commands in real time.”
“The flow matching transformer is implemented as a diffusion transformer (DiT)”
“splits these vectors into semantically meaningful fields ... each with their own metadata”
token mixer 3
“we use a simple cross-attention mechanism”
softmax attention 2
“Keeps padded action dimensions and horizon tokens out of the action encoder and the DiT self-attention.”
“Keeps left-padded VL tokens out of the action head”
normalization & residual 1
“the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).”
prediction head 1
“For the IDM model architecture, we use the Diffusion Transformer module (System 1) with SigLIP-2 vision embeddings”
multimodal architecture 10
“This cross-embodiment model takes multimodal input, including language and images, to perform manipulation tasks in diverse environments.”
“It adopts a dual-system compositional architecture, inspired by human cognitive processing”
“Robot proprioception is encoded using a multi-layer perceptron (MLP) indexed by the embodiment ID.”
“we use embodiment-specific state and action projector modules, which support different robot embodiments”
“Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.”
“camera frames are processed through a pre-trained vision transformer (SigLip2).”
“Text is encoded by a pre-trained transformer (T5)”
“The neural network architecture of GR00T N1.7 is a combination of vision-language foundation model and diffusion transformer head that…”
“We found that using middle-layer instead of final-layer LLM embeddings resulted in both faster inference speed and higher downstream…”
“built using pre-trained vision and language encoders”
auxiliary loss 2
“To enhance the model’s spatial understanding, we introduce an auxiliary object detection loss during training”
“append a linear layer atop the final vision-language embedding tokens and optimize using a squared loss”
training stability 1
“Randomly drops state inputs during training to improve generalization and reduce state-dependency.”
data sourcing 2
“EgoScale — pre-training on 20,854 hours of human egocentric video spanning 20+ task categories”
“Training on sensorized human video (ego cameras, wrist cameras, hand tracking) gives the model rich manipulation priors”
data filtering 3
“We also apply post-processing mechanisms, including filtering and re-captioning, to the generated videos.”
“we filter out generated videos that do not accurately follow the given language instructions”
“Only successful demonstrations are retained, ensuring high-quality data.”
synthetic data 5
“After training, we pseudo-label the actions given the two images ... for each step of the neural trajectories”
“we leverage the recent advancements in video generative models ... to create entire neural trajectories”
“for each frame in a trajectory segment, we annotate the bounding box of the target object using the OWL-v2 object detector”
“Videos that fail this criterion undergo re-captioning”
“DexMimicGen applies demonstration transformation and replay in simulation to expand the dataset automatically.”
data mixture & curriculum 7
“we developed techniques to effectively learn from a diverse assortment of real-world robot data, human video data, and synthetic data”
“Our way of co-training with synthetically generated and real-world data”
“we structure the VLA training corpora as a data pyramid”
“We develop an effective co-training strategy to learn across the entire data pyramid in both pre- and post-training phases.”
“During post-training, we co-train the policy with real-world trajectories with neural trajectories with a 1:1 sampling ratio.”
“by sampling training batches across this heterogeneous data mixture”
“Fine-tuning can use multiple dataset paths with mixture weighting, making multi-dataset training easier to configure.”
1 more mentioned or evaluated
synthetic data “GR00T N1 co-trained with neural trajectories consistently results in substantial gains”
filed at the root 1
“Developers and researchers can post-train GR00T N1.7 with real or synthetic data”
supervised fine-tuning 10
“We finetune WAN2.1-I2V-14B ... using LoRA on collected teleoperation trajectories”
“we fine-tune image-to-video generation models on all of our 88 hours of in-house collected teleoperation data and generate 827 hours of…”
“The language and visual backbones are frozen while the projector, diffusion model, and VLLN modules are trainable.”
“we keep the language component of the VL backbone frozen and fine-tune the rest of the model.”
“N1.7 post-trained model using the Bridge Dataset in SimplerEnv.”
“N1.7 post-trained model using the DROID Dataset.”
“N1.7 post-trained model using the Fractal Dataset in SimplerEnv.”
“N1.7 post-trained model using the LIBERO Dataset.”
“In the post-training phase, we fine-tune our pre-trained model on datasets corresponding to each single embodiment.”
“N1.7 supports fine-tuning on custom embodiments using the LeRobot dataset format.”
mid-training & continual pretraining 1
“N1.7 is pretrained on 20K hours of EgoScale human video data alongside diverse robot demonstrations.”
filed at the root 1
“The policy runs on a GPU server; a lightweight client sends observations and receives actions over ZMQ.”
filed at the root 1
“We evaluate in both simulation and real robot benchmarks”
judge 1
“we also use a commercial-grade multimodal LLM as a judge and feed the downsampled 8 frames to filter out neural trajectories that do not…”
human & real-world evaluation 2
“For sim, we automatically measure the success rate in each manipulation behavior.”
“Evaluated by human observers in the lab”
other 3
filed at the root 3
“The recorded human movements are then retargeted to humanoid actions via inverse kinematics.”
“the model can transfer manipulation priors learned from human video directly to robot control.”
“conditioning on two images (current and the future frame) within a trajectory and train to generate action chunks between the two image…”
unfiled 23
“We fine-tune our VLM together with a flow-matching ... action generation model”
“action generation model with action chunking”
“It uses an Action Cascade architecture — a dual-system design that separates high-level reasoning from low-level motor control”
“uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.”
“inputs are padded to a configurable max length before feeding into the MLP.”
“Representing actions as deltas from the current pose (rather than absolute targets) improves generalization”
“This is where task decomposition and multi-step reasoning happen.”
“We require a modality.json configuration file in the meta directory that explicitly defines the structure of state and action vectors”
“Following the V-model methodology, iterative testing and validation at both unit and system levels are essential”
“End-effector rotation actions are expressed in axis-angle representation”
“Our format explicitly specifies the representation used for rotational data”
“train with a flow-matching objective”
“Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms…”
“we instead generate latent actions by training a VQ-VAE model to extract features from consecutive image frames from videos”
“the VLA predicts compact latent action tokens that a learned whole-body controller decodes into full-body joint commands”
“Min-max normalization is applied to joint states, joint actions, end-effector state positions, and end-effector action positions and…”
“We extend the format to support multiple annotation types ... within a single dataset”
“Each task is decomposed into a sequence of object-centric subtasks.”
“These segments are then adapted to new environments by aligning them with the object’s position, preserving the relative poses between the…”
“State end-effector rotations are converted to a 6D rotation representation to avoid singularities and discontinuities in traditional Euler…”
“The arrangement of state and action vectors follows a standardized sequence”
“The encoder takes the current frame and the future frame of a video with a fixed window size and outputs the latent action”
“Data collection — VR teleoperation with SONIC for demonstration recording”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| huggingface.co/nvidia/GR00T-N1.7-3B | model card | NVIDIA | read |
| huggingface.co/blog/nvidia/gr00t-n1-7 | official blog | NVIDIA | read |
| arxiv.org/abs/2503.14734 | technical report | NVIDIA | read |
| github.com/NVIDIA/Isaac-GR00T | code repo | NVIDIA | read |