Mistral-Large-4
Mistral AIrouter replay re-executes the expert-routing decisions made at inference time during the training forward pass, eliminating divergence caused by router drift.
specific method · filed under optimization
Re-executes inference-time expert-routing decisions during training to prevent divergence from router drift.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training stability for the methods filed beside it.
2 quotes from 2 sources, one card per model, each opening with its strongest.
router replay re-executes the expert-routing decisions made at inference time during the training forward pass, eliminating divergence caused by router drift.
augmented with two optimizations designed to mitigate train–inference mismatch: router replay (Ma et al., 2025) and using FP32 for LM head