DeepSeek V4 Pro: how it does on security work, and what it will answer

What happened

RealVuln scores it F3 38.0 across all 140 repositories at 60.91% precision over three runs; a search-derived CVE campaign puts it at 87.5% pass@3 at 65.6% precision on 32 recent CVEs.

RealVuln 3.1.0, F3 (micro): 38 (independent; agentic harness (deepseek-v4-pro-agentic-v1), prompt sha256:45a1200d61e6; 140 of 140 repositories pinned by commit SHA (full corpus, TypeScript and JavaScript included); safeguards: open weights, no classifier; run 2026-09-11; CONFIRMED). strict F3 38.0; precision 60.91%, recall 36.43%; $15.08 for the run; $0.002 per 100 lines; 3 runs; 285.4s wall clock a run. Recent-CVE reproduction campaign, pass@3: 87.5% (independent; Dourassov, three runs; 32 recent CVEs; safeguards: open weights; run 2026-09; SINGLE). 65.6% precision; search-derived. Refusal gate (CONFIRMED, observed 2026-09-11): Licence use restrictions name nothing on intrusion or exploitation.