PublishersSebastian Raschka

Sebastian Raschka

1 document read from this publisher.

Inkling: A New Open-Weight 975B MoE with a Few Surprises

news analysis · third party analysis · 2026-07-16 · 7 techniques

Mixture of Experts Grouped-query attention 512-token Sliding Window Attention Learned input-dependent relative position bias Kernel-4 convolutional layers in decoder blocks Post-Embedding RMSNorm Regular Transformer decoder