How to identify a stealth AI model: tokenizer fingerprinting, API contracts and infrastructure

An anonymous model appears on OpenRouter under a codename, free to use, with no maker named. Within a week or two the operator usually says who it is. The interesting window is before that, and three classes of evidence can place a model inside it.

They are worth keeping apart, because they reach different distances. The API contract is authored by the platform. The tokenizer is produced by the inference backend. The documents around a listing are written by whoever wrote them. Only the second survives the objection that a catalogue record was copied at onboarding, and only the third can name a company.

This is the method as we ran it on Ox Alpha in August 2026 and Union Alpha in September, written so someone else can run it and check every answer against their own readings.

Freeze the catalogue before you touch anything

Save the platform’s full model list to a file, with a date in the filename, before you send a single request. Stealth entries are renamed, re-tagged and withdrawn once the cloak comes off, and every figure you publish about the listing becomes uncheckable the moment that happens.

curl -s https://openrouter.ai/api/v1/models > openrouter-models-2026-09-16.json

Do the same for any documentation page carrying the model, including the specific commit if it lives in a public repository. Two revisions of one file, captured a day apart, are what let anyone confirm later that a description changed.

Fingerprint the API contract

Every catalogue record publishes a structured description: context length, maximum completion, input and output modalities, tool support, reasoning parameters, tokenizer tag. Treat the combination as a signature and ask how many other models in the catalogue carry all of it.

On Ox Alpha this was decisive on its own. Of 418 models listed, exactly three carried the complete seven-field contract: the stealth entry, z-ai/glm-5.3, and the ~z-ai/glm-latest alias. GLM-5.2 matched two fields of seven, which showed the contract had changed at the 5.3 generation rather than being a standing template from that lab.

Know the weakness before you lean on it. The platform authors these records, so a record copied from another model at onboarding reproduces the signature with no relationship between the labs. It is a strong lead and a weak proof, and on Union Alpha it gave nothing at all: zero of 444 listed models carried that contract.

Measure the tokenizer, and measure it differentially

This is the load-bearing test, because the inference backend computes it.

Read one field, usage.prompt_tokens, with max_tokens set to 1 so the reply cannot influence anything. Send fifty strings chosen to separate vocabularies: CJK, Korean, Hebrew, Thai, Arabic, emoji including zero-width-joiner sequences and flags, combining marks, astral-plane characters, whitespace runs, digit strings, code, base64 and regex.

Then take the difference rather than the raw number. Send a string, send a longer string built from whole copies of it, and subtract the two readings. Every fixed cost cancels: the chat template, the system scaffolding, the image or tool payload if one is attached. What survives is the content’s own cost under that model’s vocabulary.

Thirty digits are the sharpest single probe available. The Llama 3 vocabulary bundles digits in threes and charges ten for thirty; the Qwen vocabulary splits them and charges thirty. Digits cannot separate Llama 3 from GLM, though, because both charge ten, so keep eight strings in reserve from emoji, Thai, Hindi and combining marks for exactly that comparison. We called an image path Llama on digits alone and had to withdraw it.

Score against a frozen corpus, with a median beside the match

A match figure alone says nothing. Score your target against every model you have measured, and publish the median alongside the best match.

Our corpus is 291 models measured on 3 September 2026. Union Alpha’s two text readings score 47 of 50 and 46 of 50 against a median of 12. Ox Alpha scored 50 of 50 against GLM-5.3, with the nearest control at 36%.

Report the whole top of the table rather than the single best row. Eight models tie at the top of Union Alpha’s first reading across three different companies, which is the result telling you honestly that it has placed a vocabulary and stopped there.

Run the control, every time

Stealth entries usually carry tokenizer: "Other" in platform metadata. If that tag made the platform fall back to one shared estimator, every model carrying it would score alike, and your agreement would be an artefact of the catalogue.

So score the whole tagged group. In our corpus 104 models carry the tag, and scored against the same reading their results run from 94% down to zero: Perplexity’s Sonar pair at the top, Upstage’s Solar Pro 4 mid-table at 32%, Inception’s Mercury 2.5 at the floor. The tag produces no shared answer, so the agreement belongs to the models. Sonar sitting at the top is the method working rather than the tag leaking, because Sonar genuinely carries the Llama 3 vocabulary.

Ox Alpha had the same control in a different shape. Kimi K3 carries the tag, exactly as Ox Alpha and GLM-5.3 did, and scored 18%.

Probe every request shape, images and tools included

Send the same strings again under a system prompt, after a prior assistant turn, with code, with a long context, with an image attached and with a tool declaration. Endpoints that route can answer different request shapes with different models.

Union Alpha is the reason this step exists. Plain text alternated between two vocabularies from byte-identical requests. Attach an image and the alternation stopped: fourteen identical readings out of fourteen, landing on a third family, and the same again under tool declarations.

Plain text alone names one family at a time. Several teams read it that way, ourselves included, and the image and tool paths are what revealed the rest of the system.

Watch how the reported numbers behave under repetition

Some providers report prompt_tokens disturbed by prompt caching, and the disturbance is additive and intermittent rather than a fixed correction you can subtract.

Hammer six strings a dozen times each and keep prompt_tokens_details.cached_tokens with every reading. If the readings split by cache state, a single-shot measurement of that endpoint is unreliable. On Union Alpha the single character x took exactly two values across 23 pooled requests, 1 and 18, and which one arrived was fully determined by the cache metadata.

Repetition is the answer. The true reading is present among the observations for every string, and repeats across separate runs recover it.

Read the paper trail

Documents are what turn a family into a company, and they are usually sitting in public.

Arithmetic on published costs. If a gateway documents worked examples with token counts and a dollar cost, those are simultaneous equations. Two uncached examples give two equations in two unknowns and return the input and output rates with nothing assumed. A cached example then forces the cached-input rate. Union Alpha’s five examples solved to $1.25 input, $0.15 cached and $6.25 output per million, which was one company’s published card.

Commit history. Documentation in a public repository carries its own revisions. Union Alpha’s Cloudflare entry described a blended system on 16 September and was rewritten on 17 September; both revisions are in the history, and a capture of each keeps the quotes checkable.

Completion IDs. Generation IDs often encode a timestamp. Union Alpha’s archived examples begin chatcmpl- followed by eight base-36 characters, and read as milliseconds they decode to five calls inside 16.479 seconds on 15 September, about 15 hours before the listing went up.

node -e 'console.log(new Date(parseInt("mu3b7o02",36)).toISOString())'

Be careful what an ID prefix proves. Because those leading characters are a clock, every ID minted by that scheme between 13 September and 8 October 2026 begins mu, so a prefix match carries an era and a shared library rather than a company signature.

Infrastructure is the strongest trace, and the hardest to check

Where two names are answered by the same backend fleet, that is close to dispositive, because a serving fleet is much harder to share by accident than an ID format or a price list.

It also carries a problem a reader cannot solve: an infrastructure observation is normally one person’s report of their own access, and nobody else can rerun it. Treat it as a source claim, say so, and weigh it beside evidence anyone can reproduce. On Union Alpha the fleet observation came from a researcher hours before the operator confirmed the name, and both were reported as what they were.

The controls that stop a false positive

Five habits do most of the work.

• Publish the median with the match. A 46 of 50 means nothing until a reader knows the corpus median is 12.
• Score the tag group. If the platform could have produced your agreement, you have measured the platform.
• Test linearity across unrelated content types. One constant fitting digits, Chinese and emoji at once is a vocabulary. One constant fitting one pair is arithmetic.
• Keep independent clues independent. Cloudflare’s pricing and its architecture wording sat on one page, so they may share one origin, and multiplying them together would have overstated the case.
• Separate the measurement from the inference built on it. A tokenizer reading is a measurement. Naming the lab behind it is a judgement laid on top, and it gets published as one.

Write the prediction down before the answer arrives

Publish a dated claim register while the model is still anonymous: each claim, its confidence, what would confirm it, what would refute it. Then score it when the reveal lands, and keep every row in the order it was written.

Ours on Union Alpha ran nine claims. Five of them rested on a single-vocabulary reading, and the reveal graded all nine at once, because the endpoint held several models and each vocabulary reading had caught a real one. Every row stays published in date order, because a register read end to end is what lets anyone judge the next call, and the next call is the only reason to keep one.

The full worked example behind every step here, with the timestamps and the raw readings, is in how we identified Ox Alpha and Union Alpha. The measurements themselves are on the model fingerprints page.