Open Models Almanac
ModelsGR00T N1.7

NVIDIA

GR00T N1.7

What GR00T N1.7’s own documents say it is built from — every method with the sentence that describes it.

Parameters
3.1B
Attention
Cosmos-Reason2-2B VLM (System 2) + flow-matching DiT action head (System 1)
Input
text+image -> action
Licence
NVIDIA Open Model License (commercial); GitHub says Apache-2.0: check
Released
2026-04-17

Against the consensus recipe

Its documents state 0 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

filed at the root 3

Diffusion Transformer (DiT) core

“A 32-layer DiT takes the VLM's output and live robot state, then denoises them into precise motor commands in real time.”

Flow-matching transformer core

“The flow matching transformer is implemented as a diffusion transformer (DiT)”

Semantically partitioned state and action vectors with field metadata used

“splits these vectors into semantically meaningful fields ... each with their own metadata”

token mixer 3

Cross-attention core

“we use a simple cross-attention mechanism”

softmax attention 2

Strict action padding mask optional

“Keeps padded action dimensions and horizon tokens out of the action encoder and the DiT self-attention.”

Strict vision-language padding mask optional

“Keeps left-padded VL tokens out of the action head”

normalization & residual 1

Adaptive Layer Normalization (AdaLN) core

“the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).”

prediction head 1

Diffusion Transformer IDM using SigLIP-2 vision embeddings used

“For the IDM model architecture, we use the Diffusion Transformer module (System 1) with SigLIP-2 vision embeddings”

multimodal architecture 10

Cross-embodiment modeling core

“This cross-embodiment model takes multimodal input, including language and images, to perform manipulation tasks in diverse environments.”

Dual-system compositional architecture core

“It adopts a dual-system compositional architecture, inspired by human cognitive processing”

Embodiment-indexed proprioception MLP core

“Robot proprioception is encoded using a multi-layer perceptron (MLP) indexed by the embodiment ID.”

Embodiment-specific state and action projectors core

“we use embodiment-specific state and action projector modules, which support different robot embodiments”

Per-embodiment action MLPs core

“Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.”

SigLIP2 pretrained vision transformer core

“camera frames are processed through a pre-trained vision transformer (SigLip2).”

T5 pretrained text encoder core

“Text is encoded by a pre-trained transformer (T5)”

Vision-language model with diffusion-transformer action head core

“The neural network architecture of GR00T N1.7 is a combination of vision-language foundation model and diffusion transformer head that…”

Middle-layer VLM representations for policy conditioning used

“We found that using middle-layer instead of final-layer LLM embeddings resulted in both faster inference speed and higher downstream…”

Pre-trained vision and language encoders used

“built using pre-trained vision and language encoders”

auxiliary loss 2

Auxiliary object-detection loss core

“To enhance the model’s spatial understanding, we introduce an auxiliary object detection loss during training”

Squared-error loss for normalized object-center coordinates core

“append a linear layer atop the final vision-language embedding tokens and optimize using a squared loss”

training stability 1

State-input dropout optional

“Randomly drops state inputs during training to improve generalization and reduce state-dependency.”

data sourcing 2

EgoScale core

“EgoScale — pre-training on 20,854 hours of human egocentric video spanning 20+ task categories”

Pre-training on sensorized human video core

“Training on sensorized human video (ego cameras, wrist cameras, hand tracking) gives the model rich manipulation priors”

data filtering 3

Filtering and re-captioning generated neural trajectories used

“We also apply post-processing mechanisms, including filtering and re-captioning, to the generated videos.”

Instruction-adherence filtering of generated videos used

“we filter out generated videos that do not accurately follow the given language instructions”

Success-based filtering of simulated demonstrations used

“Only successful demonstrations are retained, ensuring high-quality data.”

synthetic data 5

Inverse-dynamics-model pseudo-action labeling used

“After training, we pseudo-label the actions given the two images ... for each step of the neural trajectories”

Neural trajectory generation using video generative models used

“we leverage the recent advancements in video generative models ... to create entire neural trajectories”

OWL-v2 target-object bounding-box annotation used

“for each frame in a trajectory segment, we annotate the bounding box of the target object using the OWL-v2 object detector”

Recaptioning videos that fail instruction-adherence review used

“Videos that fail this criterion undergo re-captioning”

Simulation trajectory generation through demonstration transformation and replay used

“DexMimicGen applies demonstration transformation and replay in simulation to expand the dataset automatically.”

data mixture & curriculum 7

Heterogeneous training mixture core

“we developed techniques to effectively learn from a diverse assortment of real-world robot data, human video data, and synthetic data”

Co-training on synthetic and real-world data core

“Our way of co-training with synthetically generated and real-world data”

Heterogeneous training data pyramid core

“we structure the VLA training corpora as a data pyramid”

Heterogeneous-data co-training used

“We develop an effective co-training strategy to learn across the entire data pyramid in both pre- and post-training phases.”

1:1 co-training of real-world and neural trajectories used

“During post-training, we co-train the policy with real-world trajectories with neural trajectories with a 1:1 sampling ratio.”

Heterogeneous data-mixture batch sampling used

“by sampling training batches across this heterogeneous data mixture”

Weighted multi-dataset fine-tuning optional

“Fine-tuning can use multiple dataset paths with mixture weighting, making multi-dataset training easier to configure.”

1 more mentioned or evaluated
Co-training with generated neural trajectories evaluated

synthetic data “GR00T N1 co-trained with neural trajectories consistently results in substantial gains”

filed at the root 1

Post-training with real or synthetic data optional

“Developers and researchers can post-train GR00T N1.7 with real or synthetic data”

supervised fine-tuning 10

Low-Rank Adaptation used

“We finetune WAN2.1-I2V-14B ... using LoRA on collected teleoperation trajectories”

Fine-tuning video generation models on robot teleoperation data used

“we fine-tune image-to-video generation models on all of our 88 hours of in-house collected teleoperation data and generate 827 hours of…”

Freeze language and visual backbones during fine-tuning used

“The language and visual backbones are frozen while the projector, diffusion model, and VLLN modules are trainable.”

Freeze the language component of the vision-language backbone used

“we keep the language component of the VL backbone frozen and fine-tune the rest of the model.”

Post-training on the Bridge Dataset in SimplerEnv used

“N1.7 post-trained model using the Bridge Dataset in SimplerEnv.”

Post-training on the DROID Dataset used

“N1.7 post-trained model using the DROID Dataset.”

Post-training on the Fractal Dataset in SimplerEnv used

“N1.7 post-trained model using the Fractal Dataset in SimplerEnv.”

Post-training on the LIBERO Dataset used

“N1.7 post-trained model using the LIBERO Dataset.”

Single-embodiment post-training fine-tuning used

“In the post-training phase, we fine-tune our pre-trained model on datasets corresponding to each single embodiment.”

Fine-tuning on custom embodiments with LeRobot-format datasets optional

“N1.7 supports fine-tuning on custom embodiments using the LeRobot dataset format.”

mid-training & continual pretraining 1

Human-video pretraining used

“N1.7 is pretrained on 20K hours of EgoScale human video data alongside diverse robot demonstrations.”

filed at the root 1

Server-client policy inference optional

“The policy runs on a GPU server; a lightweight client sends observations and receives actions over ZMQ.”

filed at the root 1

Simulation and real-robot benchmark evaluation used

“We evaluate in both simulation and real robot benchmarks”

judge 1

Multimodal LLM instruction-adherence judging used

“we also use a commercial-grade multimodal LLM as a judge and feed the downsampled 8 frames to filter out neural trajectories that do not…”

human & real-world evaluation 2

Automatic success-rate measurement used

“For sim, we automatically measure the success rate in each manipulation behavior.”

Human-observer evaluation of real-robot tasks used

“Evaluated by human observers in the lab”

filed at the root 3

Inverse-kinematics retargeting used

“The recorded human movements are then retargeted to humanoid actions via inverse kinematics.”

Transfer of human-video manipulation priors to robot control used

“the model can transfer manipulation priors learned from human video directly to robot control.”

Two-frame-conditioned inverse dynamics model used

“conditioning on two images (current and the future frame) within a trajectory and train to generate action chunks between the two image…”

unfiled 23

Flow matching core

“We fine-tune our VLM together with a flow-matching ... action generation model”

Action chunking core

“action generation model with action chunking”

Action Cascade dual-system architecture core

“It uses an Action Cascade architecture — a dual-system design that separates high-level reasoning from low-level motor control”

Flow-matching action transformer core

“uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.”

Padding variable-dimension proprioception inputs to a configurable maximum length core

“inputs are padded to a configurable max length before feeding into the MLP.”

Relative end-effector action representation core

“Representing actions as deltas from the current pose (rather than absolute targets) improves generalization”

Task decomposition and multi-step reasoning core

“This is where task decomposition and multi-step reasoning happen.”

Explicit modality configuration for state and action vectors default

“We require a modality.json configuration file in the meta directory that explicitly defines the structure of state and action vectors”

V-model iterative unit- and system-level testing and validation used

“Following the V-model methodology, iterative testing and validation at both unit and system levels are essential”

Axis-angle representation for end-effector rotation actions used

“End-effector rotation actions are expressed in axis-angle representation”

Explicit rotational-data representation specification used

“Our format explicitly specifies the representation used for rotational data”

Flow-matching training objective for the IDM used

“train with a flow-matching objective”

Implementing guardrails and safety mechanisms before deployment used

“Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms…”

Latent-action representation learned from video used

“we instead generate latent actions by training a VQ-VAE model to extract features from consecutive image frames from videos”

Latent-action VLA with learned whole-body controller decoding used

“the VLA predicts compact latent action tokens that a learned whole-body controller decodes into full-body joint commands”

Min-max normalization of robot states and actions used

“Min-max normalization is applied to joint states, joint actions, end-effector state positions, and end-effector action positions and…”

Multiple annotation types in a single dataset used

“We extend the format to support multiple annotation types ... within a single dataset”

Object-centric subtask decomposition for simulation demonstrations used

“Each task is decomposed into a sequence of object-centric subtasks.”

Object-position alignment for adapting demonstration segments used

“These segments are then adapted to new environments by aligning them with the object’s position, preserving the relative poses between the…”

Represent end-effector rotations in 6D for state normalization used

“State end-effector rotations are converted to a 6D rotation representation to avoid singularities and discontinuities in traditional Euler…”

Standardized ordering of state and action vectors used

“The arrangement of state and action vectors follows a standardized sequence”

VQ-VAE training for video-derived latent actions used

“The encoder takes the current frame and the future frame of a video with a fixed window size and outputs the latent action”

VR teleoperation for robot demonstration collection used

“Data collection — VR teleoperation with SONIC for demonstration recording”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
huggingface.co/nvidia/GR00T-N1.7-3Bmodel cardNVIDIAread
huggingface.co/blog/nvidia/gr00t-n1-7official blogNVIDIAread
arxiv.org/abs/2503.14734technical reportNVIDIAread
github.com/NVIDIA/Isaac-GR00Tcode repoNVIDIAread
↑