Model Security Capability

What the numbers are. 30 models with 50 score rows, 18 of them from an independent harness, as of 19 September 2026. A score is a benchmark result: RealVuln's F3, which weights recall nine to one over precision, on a pinned corpus of real repositories; Cybench's end-to-end solve rate on professional capture-the-flag tasks; and the vendors' own security tables, with the labs' published refusal rates beside them. A confidence marker on a row is carried from the research note: CONFIRMED, SINGLE, CONTESTED or UNVERIFIED.

Where they come from. RealVuln pins every repository by commit, hashes the prompt and publishes what each run cost; the board reads version 3.1.0, dashboard generated 11 September 2026. Cybench is 40 tasks, and the vendor's task count is printed beside every row; the leaderboard CSV was read 11 September 2026. Refusal figures come from each lab's own contract: Anthropic's typed refusal object, OpenAI's Model Spec and Daybreak tiers, Google's programmes and the open-weight licences.

What they say. On RealVuln GPT Daybreak Blue leads the model rows at F3 79.5 across all 140 repositories, GPT-5.6 Sol 74.7 and Claude Opus 5 67.7 on the 66-repository Python subset. On Cybench Claude Mythos Preview scores 100% on 35 of 40 tasks. Anthropic's classifiers block 7.0% of defensive discovery on Claude Fable 5.1 against 90.0% on Fable 5 at launch.

The caveats. A vendor-card row and an independent row are two different measurements, so the board labels every row with its provenance and charts the two kinds separately. Vendor scaffolds run roughly 15 to 30 points above a shared harness. RealVuln's micro figures score the repositories a model ran and its strict figures score all 140; a 66/140 row's strict F3 is lower and sits on the model's record. Kolega's own devsec-max stack scores 84.4 with no cost published and is left off the model bars.

The dates on the board. A row is dated by the lab's own release date, which 20 of the 30 models carry; the rest are dated by the board's own measurement, 19 Sept 2026, and follow the dated releases in the default order.

Refresh it. Re-pull the RealVuln dashboard monthly and read the aggregates block; re-read the Cybench CSV on each frontier release:

curl -s -o rv.json https://raw.githubusercontent.com/kolega-ai/Real-Vuln-Benchmark/main/reports/dashboard.json
python3 -c "import json; d=json.load(open('rv.json'))
for k,v in d['aggregates'].items():
    c=v.get('cost') or {}; m=v['micro']
    print(k, v['repos_scored'], m['f3_score'], c.get('cost_per_run'), c.get('cost_per_100_loc'))"