Open Models Almanac

about · how the almanac is made

Every method, with the sentence that describes it.

A method only exists here if a technical report describes its use in a sentence that can be quoted. Extraction reads each source, names the methods it substantively treats, and files each one into a hand-curated taxonomy. The Almanac portion of this site tries to provide a more self-contained read while still being true to the extracted facts.

3 299
methods
5 120
quotes
530
unfiled
no taxonomy path
6
taxonomy proposals
awaiting review

Who maintains this

I’m Yannick Versley, and I maintain the Open Models Almanac as a way for others to learn about what goes on in the world of LLMs/VLMs, in (almost) direct from the published practice of the labs that train them. While not all frontier lab practices are useful outside a large-scale lab setting, providing a contextualized view of what’s used today is a much more appropriate approach than an academic textbook (soon outdated!) or reading reports front-to-back (not self-contained!).

Let’s also celebrate open-weight models - also and especially the ones that are more open with their recipes - as positive force in the world for education, and for the self-determined, flexible use of large language models outside a handful of commercial APIs. Knowing more about what actually happens inside these models helps clarify the fundamental problems they still have to solve.

Homepage · LinkedIn · Google Scholar

How a page here is produced

  1. Ranking. OpenRouter’s weekly open-weight token ranking is refreshed automatically; new entrants land in a review queue.
  2. Curation. A human adds the model’s sources — blog post, technical report, model card, repository — to a checked-in list.
  3. Fetch. Each source is fetched and reduced to text. The curation labels (which model, which publisher) are withheld from the extractor, so what it finds is not what it was told.
  4. Extraction. Per source: what kind of document it is, then every method it substantively treats, each with a required quote, an adoption status, and the artifact it applies to.
  5. Filing. Methods are grouped, filed into the curated taxonomy, and merged into named methods. Nothing invents a taxonomy path.
  6. Relations. Per source, which of its methods are used for which others — an iteration that implements an optimizer, a gate that is a form of gated attention. An edge is shown only between two methods with pages, only with a cue found in the source, and only after rules against recipe lists, reversed direction and goals posing as methods. It is a second claim beside the quotes, never a new method.
  7. Corrections. A hand-kept, checked-in file fixes what the pipeline gets wrong: a word the PDF text ran together is repaired before extraction (and the quote says so), a misread method or relation is removed, and pairs of taxonomy branches that cannot contain each other are declared once. Every entry records why.
  8. Code checks. Separately, each model’s modeling code and config.json — from Hugging Face Transformers, the model repository, or vLLM, pinned to a commit — are checked against a short, curated list of architecture features. The verifier is not told the model’s name or what its sources say, and a verdict is shown only with the config values and source lines it cites that were found in the files.

Sources

Where the quotes come from

233 sources from 64 publishers, most method mentions first.

Glossary

The words used here

One noun each, on every page.

Method
A named design or training choice that a source describes. Each has its own page; older links say “technique”, and only the URLs still do.
Quote
The sentence from a source that a claim rests on. A method without one is not published.
Source
One document that was read: a technical report, blog post, model card, repository, documentation page or article.
Lab
The organisation that makes a model.
Publisher
Whoever put out a source: the model’s lab, or a third party. Used only under Sources; everywhere else the organisation behind a model is its lab.
Adopted
A model’s sources say it uses the method, by default or as an option.
Discussed
A model’s sources mention or evaluate the method without saying it is in use.
Not used
A model’s sources say it does not use the method. A model whose sources are silent gets no mark at all.
Area
A top-level branch of the map: a stage of the pipeline, or a part outside it.
Node
Any place inside an area where methods are filed.

Taxonomy proposals

What the corpus wanted and the curated outline does not have. These are not published as methods until a human promotes them.

ParentProposedRationale
post-trainingrobotics action representation & controlThe groups repeatedly describe robot/VLA action representations and generation: chunked trajectories, continuous action experts trained by conditional flow matching or regression,
model architecture › token mixershort convolutionThese groups use short, typically gated, temporal convolutions as a sequence-mixing mechanism, sometimes combined with sliding-window and global attention. The existing token-mixer
optimizationautomated hyperparameter and experiment searchThese methods search or infer good training settings by iteratively tuning hyperparameters or extrapolating loss landscapes from proxy runs. The optimizer and learning-rate schedul
inference & servingprovider routing and failoverThese groups route requests among upstream model providers and retry on a healthy alternative when a provider fails. Serving parallelism concerns distributing model computation, an
data curationrobotics action and state normalizationThese methods compute, merge, and apply per-dataset, per-robot, per-timestep, or percentile statistics to scale robot states and actions into a consistent representation. Existing
evaluationbenchmark contamination detectionThese groups detect possible overlap between benchmark questions and training or evaluation data using retrieval, lexical/semantic similarity, and windowed matching. The benchmark

Credits and licences

Model ranking: Source: OpenRouter (openrouter.ai/rankings), as of 2026-11-02T16:59:23.313Z. Licensed CC BY 4.0.

The text of a source belongs to its publisher; quotes here are short excerpts shown with attribution and a link to the original.