PublishersKimi Team

Kimi Team

1 document read from this publisher.

k3 tech report

technical report · 153 techniques

Hybrid Attention Multi-Teacher On-Policy Distillation Supervised fine-tuning Muon LatentMoE Quantization-Aware Training Attention Residuals (AttnRes)Normalized LatentMoE Preserved thinking history mode Kimi Delta Attention Auxiliary-loss-free load balancing Cosine decay Partial rollout Per-Head Muon Quantile Balancing Warmup-Stable-Decay Blind side-by-side expert evaluation MoonEP Updated hyperparameter scaling law Gated Multi-Head Latent Attention SiTU-GLU Block AttnRes Co-located RL training Histogram-based quantile estimation KDA-aware prefix-cache management MiniTriton No Position Encoding Progressive context extension Synchronization-free static-shape MoE execution Zero-copy fused token permutation and unpermutation 2×2 pixel-shuffle downsampling AgentENV Agentic Generative Reward Model Agentic knowledge graph construction autonomous chip design and verification with open-source EDA tools Autonomous Execution Tasks (AET)Block-wise FP8 activation quantization with offload Budget-based verbosity control Cache-Based Pipeline Communication Co-located long-context agentic RL system