SourcesGemma Team, Google DeepMind
publisher · 1 source
Gemma Team, Google DeepMind
1 source read from this publisher, newest first, with the methods each one describes.
Gemma 4 Technical Report
technical report · 2026-07-02 · 29 methods
Mixture of expertsThinking modeHybrid Sliding-Window and Global AttentionMulti-Token PredictionQuantization-Aware TrainingBenchmark decontaminationEncoder-free multimodal architecturePer-Layer Embeddings (PLE)Evaluation without safety filtersMobile quantizationRaw audio projection into LLM embedding space2D coordinate-based positional embeddingsAxial 2D-RoPEClustered top-K projectionFrozen encoders during pre-trainingIn-context attribution, hedging, and refusal data subsetsKey-as-value reuse in global attention layersKV cache sharingPathways data-replica reductionPer-Block Scalar Scalingpost-training data filtering for safety and factualityQ4_0SentencePieceSeparate end tokens for pretraining and instruction-tuned modelsSingle-matmul vision projectionSlice-Granularity ElasticityUniversal Speech Model-based audio encoderVariable aspect-ratio image handlingZeRO-3 optimizer-state sharding