SourcesAleph Alpha
publisher · 4 sources
Aleph Alpha
4 sources read from this publisher, newest first, with the methods each one describes.
Kolibri Has Landed: A Sovereign Open-Weight Model — Aleph Alpha
first party release · official blog · 2026-10-03 · 26 methods
Mixture of expertsThinking modeSupervised fine-tuningHybrid Sliding-Window and Global AttentionAsynchronous reinforcement learningLong-context extension via continued pretrainingMuonUniBPEAutomatic training recovery after failuresExact Quantile BalancingGerman document rephrasing for data augmentationAbstention training with explicit negative examplesAutomated end-to-end training and evaluation on every code changeAutomatic checkpoint evaluation across capability benchmarksBilingual English-German tokenizerDomain-specific evaluation suitesGrounding and hallucination evaluationLanguage-specific filter retuningLarge-scale reinforcement learning for long-horizon tasksMany small expertsMerlin-Arthur synthetic negative-example generation procedureRandomized training environments for real-world robustnessRemoving synthetic long documents from the long-context mixReproducible training runs tied to version-control commitsSynthetic-data training environments for domain-specific skillsVersion-controlled training pipeline and recipe
Aleph-Alpha/Kolibri-1 · Hugging Face
first party release · model card · 2026-01-25 · 58 methods
Mixture of expertsThinking modeHybrid Sliding-Window and Global AttentionAsynchronous reinforcement learningAdamWQuantization-Aware TrainingBenchmark decontaminationGlobal-norm gradient clippingCorpus deduplicationMergeMixMerlin–Arthur protocolOn-Policy Self-DistillationUniBPELanguage-consistency rewardLoad-error gradient injectionMergeMix expert-checkpoint mergingQK NormalizationCausal next-token predictionDecoupled weight decayGerman-language reasoning post-trainingMask loss on incorrect tool calls and terminal actionsModel soupingPersonally identifiable information redactionPII-redaction-threshold document filteringRetrieval-augmented question-answering trainingApplication-layer logging of model outputsApplication-level output validationAvoid unsupervised use of foundation models in high-stakes environmentsBilingual German-English data curationEmpirical pre- and mid-training data-mix selectionGerman morphology-aware tokenizationGreedy bottom-up BPE mergingHallucination mitigation through abstention and context-sensitive RLHarmful and unsuitable training-data filteringHarness randomization for agentic trainingHermes-style tool callingHeuristic web-document filtering and cleanupHuman review before acting on model outputsLanguage-specific fastText routing and filteringLong-context post-training with document question answering
tech report
technical report · 354 methods
Mixture of expertsThinking modeHybrid Sliding-Window and Global AttentionAsynchronous reinforcement learningLong-context extension via continued pretrainingSliding Window AttentionMuonShared expertTop-k expert routingAdamWBenchmark decontaminationGlobal-norm gradient clippingCategory-specific optimizer assignmentStatic YaRNCheckpoint mergingReinforcement Learning with Verifiable RewardsGreedy decodingMergeMixMerlin–Arthur protocolNo Position EncodingOn-Policy Self-DistillationUniBPEAsynchronous checkpointingAutomatic training recovery after failuresBackfilling missing reasoning between tool callsExact Quantile BalancingExpert granularityKL-regularized data-mixture optimizationLanguage-consistency rewardLinear interpolation from pre-training mix to mid-training mixLoad-error gradient injectionMergeMix expert-checkpoint mergingQK NormalizationQK-ClipReasoning-Prefill DistillationSynthetic data generationUser-simulated conversational continuationAdamAdaptive RL prompt and environment curriculumAssistant-only loss
aleph-alpha-inference
software documentation · code repo · 4 methods