AI Desk · AI Fingerprinting

Tokenizer Atlas

Explore the tokenizer-family structure underlying the AI model ecosystem: which models read text with the same vocabulary, where the families sit, and which entries share ancestry their labels never mention.

Models measured: 337
Complete fingerprints: 314
Distinct fingerprints: 69
Tokenizer groups: 56
Groups with 2+ models: 31
As at: 8 October 2026

How to read this page: a tokenizer is the fixed vocabulary a model chops text with, and it cannot change without retraining, so it works like a fingerprint. Of the 337 models measured, 314 returned all fifty rows clean; only those carry an exact signature and appear in the groups below. A group gathers the signatures the evidence shows to be one tokenizer.

The constellation

One star cluster per tokenizer group; size follows membership

Tap or hover a cluster for its family; click to jump to its members. Position and distance on the canvas carry no meaning: cluster size follows membership, and only the grouping itself is data. The full listing below carries everything the canvas shows, so nothing depends on it.

The fingerprint matrix

One column per major group, one row per test string. A bright row is a string that tells the vocabularies apart; a faded row is one they nearly agree on.

Each cell is the marginal token cost of that string under that group's exemplar, shaded within its row from the row's lowest count to its highest. Exact values sit in the cell tooltips, and two models are compared string by string on the bench.

Family explorer

One card per inferred tokenizer group with two or more models; unique signatures sit in their own section beneath. Signatures are grouped when the evidence shows one vocabulary behind different serving templates.

Qwen · group Afg_920d45ce · 42 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_5410b38e · 41 models template variant tk_920d45ce · 1 model

and 34 more

GPTfg_d1411c9d · 41 models · 1 measured signature · 1 inferred tokenizer

and 33 more

Qwen · group Bfg_c5c0a8ed · 25 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_181907f2 · 22 models template variant tk_c5c0a8ed · 3 models

and 17 more

Mistral · group Afg_e0b4c23e · 24 models · 3 measured signatures · 1 inferred tokenizerexact signature tk_0da8a999 · 22 models template variant tk_b0a747db · 1 model template variant tk_e0b4c23e · 1 model

and 16 more

Gemini · group Afg_a15512c7 · 23 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_59ca250f · 22 models template variant tk_a15512c7 · 1 model

and 15 more

DeepSeek · group Afg_869dc1cf · 17 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_73c4f0dc · 14 models template variant tk_869dc1cf · 3 models

and 9 more

Llama3fg_066351de · 15 models · 3 measured signatures · 1 inferred tokenizerexact signature tk_6588ed5b · 9 models template variant tk_47615f2d · 5 models template variant tk_066351de · 1 model

and 7 more

Z.ai · group Afg_c018f9f2 · 11 models · 3 measured signatures · 1 inferred tokenizerexact signature tk_133b086c · 9 models template variant tk_1d05586d · 1 model template variant tk_c018f9f2 · 1 model

and 3 more

MiniMaxfg_9f84b5fb · 9 models · 1 measured signature · 1 inferred tokenizer

and 1 more

Qwen · group Cfg_e60ca14a · 7 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_06ffa14a · 6 models template variant tk_e60ca14a · 1 model

InclusionAIfg_fc5a4585 · 7 models · 1 measured signature · 1 inferred tokenizer

Z.ai · group Bfg_3eaaf3e0 · 7 models · 1 measured signature · 1 inferred tokenizer

OpenAI cl100kfg_ba918336 · 6 models · 1 measured signature · 1 inferred tokenizer

ByteDance Seedfg_25bf52d0 · 6 models · 1 measured signature · 1 inferred tokenizer

Moonshot AI · group Afg_ea6068bf · 5 models · 1 measured signature · 1 inferred tokenizer

Claude · group Afg_2d06485c · 5 models · 1 measured signature · 1 inferred tokenizer

Tencent · group Afg_2b541395 · 4 models · 1 measured signature · 1 inferred tokenizer

Llama2fg_5348f9f6 · 3 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_5348f9f6 · 2 models template variant tk_03ecdf53 · 1 model

Upstagefg_865d84ce · 3 models · 1 measured signature · 1 inferred tokenizer

Nova · group Afg_76e424dd · 3 models · 1 measured signature · 1 inferred tokenizer

Mistral · group Bfg_ddc193e9 · 3 models · 1 measured signature · 1 inferred tokenizer

Llama4fg_20b9d005 · 3 models · 2 measured signatures · 1 inferred tokenizerexact signature tk_59e2520c · 2 models template variant tk_20b9d005 · 1 model

Grok · group Afg_a17bbc1d · 3 models · 1 measured signature · 1 inferred tokenizer

Grok · group Bfg_cb7f8e86 · 3 models · 1 measured signature · 1 inferred tokenizer

Cohere · group Bfg_caaaa531 · 2 models · 1 measured signature · 1 inferred tokenizer

Cohere · group Afg_2729e470 · 2 models · 1 measured signature · 1 inferred tokenizer

Gemmafg_ba03436b · 2 models · 1 measured signature · 1 inferred tokenizer

Aion Labs · group Bfg_d3c6ae80 · 2 models · 1 measured signature · 1 inferred tokenizer

Unlabelled groupfg_b6cc7610 · 2 models · 1 measured signature · 1 inferred tokenizer

Aion Labs · group Afg_13620098 · 2 models · 1 measured signature · 1 inferred tokenizer

Claude · group Bfg_310f7ba9 · 2 models · 1 measured signature · 1 inferred tokenizer

When one family name appears on several groups (Mistral has 5; Tencent has 4; Qwen has 3; Moonshot AI has 3; Nova has 3; Aion Labs has 3; Gemini has 2; DeepSeek has 2; Z.ai has 2; Claude has 2; Grok has 2; Cohere has 2; NVIDIA has 2; Poolside has 2), those are distinct measured vocabularies from the same broader model family: tokenizer generations, lettered group A, B and onwards in size order. The letters are ours; the evidence does not name the generations.

One of a kind, so far

A unique EXACT fingerprint does not take a model out of its family: the nearest relative, and how close it comes, is shown on each row

A model lands here when its exact 50-number fingerprint has no twin yet, which usually means a vocabulary revision inside its own family (Qwen3.8 sits close to the Qwen line without matching it exactly) or relatives that are still unmeasured.

Reading the Atlas

A group gathers every model whose measured fingerprints are the same vocabulary: exact signature matches, plus signatures one uniform template shift apart. Group membership is lineage evidence, not release identification, and the methodology records exactly how much the fingerprints can distinguish. The Atlas's working face is the unknowns queue: what is still unplaced, and what measurement has already placed.