Methodology
We publish the evidence, including when we are uncertain. This page records how every fingerprint is measured, the controls that gate publication, and the limits of what the method can see.
The measurement
A model's tokenizer is unusually useful for family attribution because it is fixed at training time. A name, a system prompt, a price or a model card can change overnight; the vocabulary a model reads with cannot change without retraining.
Each model is sent 50 adversarial strings: CJK, Korean, Hebrew, Thai and Arabic text, emoji with skin tones, ZWJ sequences and flags, combining marks, astral-plane characters, whitespace runs, digit strings, code, base64 and URLs, chosen to maximise disagreement between tokenizers. The reply is discarded; only the reported prompt-token count is read. The model's fixed overhead (its chat template, any hidden system prompt) is measured with a single-token baseline and subtracted, leaving the marginal token cost of each string under that model's tokenizer. Those 50 numbers are the fingerprint.
Two models sharing a tokenizer agree on every string. Two models with different tokenizers diverge on the high-information strings within a handful of samples. Because each model is measured once and compared offline, cost scales with the number of models rather than the number of pairs.
One measurement, worked through
Every figure below is read straight from the published dataset- 1
Send OpenAI: GPT-4o-mini a prompt of a single token. The endpoint reports 8 prompt tokens. Its own fixed cost, the chat template and whatever the provider prepends before the text, is 7 tokens.
- 2
Now send it one test string, five skin-tone emoji. The endpoint reports 20.
- 3
Take the overhead off: 20 − 7 = 13. That is what the string costs under the vocabulary doing the counting, and it is the figure the user is billed on.
- 4
Repeat for all 50 strings. Those 50 numbers are the fingerprint. Nothing the model writes is read at any point.
Here is why that is worth doing: three models, each from a different tokenizer group, on the 6 test strings out of 50 where they part company hardest. Each cell is the marginal cost after that model's own overhead has come off, so the columns are directly comparable.
| Test string | GPT-4o-mini | Llama 3.3 70B Instruct | Mistral Nemo |
|---|---|---|---|
| five skin-tone emoji | 13 | 30 | 40 |
| three joined emoji | 33 | 45 | 57 |
| thirty digits | 10 | 10 | 30 |
| one word repeated | 22 | 21 | 41 |
| four flag emoji | 16 | 24 | 32 |
| profession emoji | 20 | 26 | 35 |
Two models with an exact signature match return the same number in every one of these rows and in the 44 rows not shown. A vocabulary served behind a different chat template is a weaker result, not an exact match: it can sit one token out on a handful of rows, and the harness records those as a boundary-linked group rather than pretending they are identical. The full vectors are on every measured model's own page.
Reproduce this
Everything needed to re-run the measurement is published. Send the 50 strings below to any endpoint, read only the reported prompt-token count, subtract the model's own fixed overhead, and compare against the published vectors.
- Method version
- v1-50
- Dataset generated
- 18 September 2026
- Catalogue snapshot
- 16 September 2026
The test vectors and the measured fingerprints are not published as files at the moment. If you want to check a specific model against your own measurement, ask us and we will send you its numbers.
What the harness refuses to trust
Prompt caching. A provider whose token accounting is disturbed by caching reports marginals that are negative or impossibly large. Every row outside the plausible window is marked corrupt and excluded; a model with eight or more corrupt rows is declared unmeasurable rather than published wrong.
Variable overhead. The baseline is measured before and after the string set. If the two disagree, the subtraction is invalid and the run is discarded.
Provider routing. The same named model can be served by providers whose chat templates differ. Calls are pinned to the first provider observed; when routing moves anyway, a separate baseline is measured for the new provider before any of its counts are used, and every observation records which providers served it.
Template boundary shifts. A serving template that glues the test string to an adjacent character shifts affected rows by exactly one token. Two signatures that differ by a uniform single-token shift are treated as one tokenizer behind two templates, and the link is recorded rather than silently merged.
Routers and aliases. A router is not one stable model and is excluded. An alias entry points at its target's record. A serving variant inherits its base model's measurement instead of spending its own.
The endpoint is not the weights. When a fingerprint contradicts a label, the model's own published tokenizer, where one exists, is downloaded and run over the same fifty strings offline. That separates what the MODEL reads with from what the SERVING layer counts with: an endpoint whose accounting disagrees with its own model's repository is a serving-layer finding, and is recorded as such on the model's page.
The controls
Verdicts from the current dataset, 18 September 2026| Control | Requirement | Current result | Verdict |
|---|---|---|---|
| Positive | Models declared as one family (Llama3, several vendors) must measure as one tokenizer group | 12 models · 11 in the majority group · the exception: deepseek/deepseek-r1-distill-llama-70b, DeepSeek's own cross-family distill, whose serving-side accounting is documented on its page | Pass |
| Negative | Known-unrelated models must diverge, or the method is reading a platform estimator rather than the model | z-ai/glm-5.3 and openai/gpt-5-nano: 18/50 · z-ai/glm-5.3 and meta-llama/llama-3.3-70b-instruct: 22/50 · openai/gpt-5-nano and meta-llama/llama-3.3-70b-instruct: 25/50 | Pass |
| Consistency | The same model measured on different days must reproduce exactly | GLM-5.3: 50/50 identical across runs on different days | Pass |
The standing rule: if the controls fail, attribution results are not published. The gate is enforced in the pipeline that builds this site, and the current verdicts render from its output.
Resolving power
A fingerprinting system must publish how much it can actually distinguish. 280 models carry a validated measurement; 262 of those returned all fifty rows clean, and only that subset carries an exact signature (the remaining 18 are compared row by row on their clean rows). The corpus currently holds 65 distinct signatures in 51 tokenizer groups. Tokenizer evidence separates groups, never individual releases within a group: that is what the claims hierarchy is for.
The claims hierarchy
Four claims, in rising order of strength, each needing strictly more evidence:
- Observed fact. Two models returned the same measured vector.
- Relationship inference. The models likely share tokenizer lineage.
- Family attribution. The unknown model is consistent with a named family.
- Exact model attribution. This endpoint is a specific release. The measurements on this site never make this claim on fingerprint evidence alone.
Confidence is rule-based rather than numeric: very high needs an exact fingerprint match plus a second signal of a different kind (in the current dataset, the API serving contract) with nothing contradictory; high needs one very strong signal; moderate and low record weaker evidence. The two signal kinds measure different properties of the endpoint, though both pass through the same serving platform, which is why neither one alone reaches very high. Percentages would imply calibration the system does not yet have, so it does not print them. Every confidence badge on a record carries its reason beside it.
Scope and limits
Measurements are taken against public endpoints whose operator invites queries, at published prices, within provider rate limits, with no user data involved. The tooling is not for pointing at anything the operator is not entitled to query.
Known limits: a tokenizer can be reused across a lab's generations and can in principle be adopted by another organisation, so a shared fingerprint is lineage evidence rather than identification. Reported token counts depend on provider accounting, which is why the controls exist and why unmeasurable is an honest verdict. Behavioural signals change with prompts and configuration, so they are treated as supporting evidence only, and the current dataset does not use them at all.
The measurement harness, the frozen catalogue snapshots and every raw observation are committed to the YFarmX repository, so any published figure can be reproduced against its dated source. Method version: v1-50, 50 strings.