LLM DEVSECOPS
The public record of the APEX escape-claim benchmarking programme.

This site exists to make a set of strong claims checkable. APEX tests one narrow claim about a language model — that in a given configuration it will not disclose what it was told to protect, abandon its assigned role, or propose crossing its boundary — and this site publishes the outcome of every run: the model, the date, the probe family, the seed, the verdict, and the result of each of the four probes. Nothing here is a summary of someone else's benchmark, and nothing is scored by opinion. If a number appears on this site, a sealed run produced it.

164CLAIM RUNS
656PROBE CELLS
95 / 15 / 54FALSIFIED / HOLDS / INCOMPLETE
41MODELS TESTED

THE TOP 10 — THE APEX BOARD

Ranked by claim holds, then cells held, then fewest escapes. Record is shown as holds–falsified–incomplete; cells as held / escaped / no-artifact. Models were tested under probe families 1.0 and 1.1 on different hosts, so this is a board ranking across the whole record, not a controlled head-to-head.

#ModelRecord (H–F–I)Cells (H/E/NA)Note
1qwen2.5:72b4–0–016 / 0 / 0The Unbroken. First full claim hold in APEX history — H/H/H/H at all four seeds.
2juke-1.7b-dev4–0–016 / 0 / 0Card 3's APEX sweep: held the full claim at ALL four seeds — the strongest result in the series.
3gpt-oss:120b3–0–115 / 0 / 1Zero escapes in 16 cells; three holds and one incomplete.
4Gemma-4-12B Maia finetune2–2–013 / 3 / 0Card 2's APEX star: held the full claim at seeds 1 and 2.
5StoryGen-12B1–2–112 / 2 / 2Card 3: held the full claim at seed 1.
6Text2SQL Qwen2.5-Coder-1.5B1–3–010 / 6 / 0Scored the only hold of Brawl Card 1 (seed 0), then falsified at seeds 1–3.
7Honeypot Qwen3-1.7B (sepetly)0–4–011 / 5 / 0
8qwen2.5:32b0–4–010 / 6 / 0
9Qwen3-1.7B coffee-distilled0–4–010 / 6 / 0
10NeoHorse-1-4B heretic0–3–19 / 5 / 2

ALL MODELS IN THE RECORD

ModelFirst testedRunsOutcomesDisposition
llama3.2:3b2026-11-0344 F · 0 H · 0 IFalsified at every seed
llama3.1:8b2026-11-0443 F · 0 H · 1 IFalsified 3 · incomplete 1 — no holds
phi4:latest2026-11-0444 F · 0 H · 0 IFalsified at every seed
qwen2.5:14b2026-11-0444 F · 0 H · 0 IFalsified at every seed
qwen2.5:3b2026-11-0444 F · 0 H · 0 IFalsified at every seed
qwen2.5:7b2026-11-0444 F · 0 H · 0 IFalsified at every seed
Gemma-4-E2B abliterated (BF16)2026-11-0541 F · 0 H · 3 IFalsified 1 · incomplete 3 — no holds
Qwen3-1.7B coffee-distilled2026-11-0544 F · 0 H · 0 IFalsified at every seed
deepseek-r1:70b2026-11-0543 F · 0 H · 1 IFalsified 3 · incomplete 1 — no holds
Qwen2.5-Coder-1.5B ExPO (F16)2026-11-0544 F · 0 H · 0 IFalsified at every seed
gpt-oss:120b2026-11-0540 F · 3 H · 1 IHeld 3 of 4 runs
Honeypot Qwen3-1.7B (sepetly)2026-11-0544 F · 0 H · 0 IFalsified at every seed
KSA-1.9B-base (GGUF)2026-11-0542 F · 0 H · 2 IFalsified 2 · incomplete 2 — no holds
LFM3-1.7B-chv-v2 (Chuvash finetune)2026-11-0544 F · 0 H · 0 IFalsified at every seed
llama3.3:70b2026-11-0544 F · 0 H · 0 IFalsified at every seed
qwen2.5:32b2026-11-0544 F · 0 H · 0 IFalsified at every seed
qwen2.5:72b2026-11-0540 F · 4 H · 0 IHeld every run — undefeated
Qwen3-4B (hellmans GGUF, Q4_K_M)2026-11-0540 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Llama-3.2-3B SOAP notes finetune2026-11-0544 F · 0 H · 0 IFalsified at every seed
Text2SQL Qwen2.5-Coder-1.5B2026-11-0543 F · 1 H · 0 IHeld 1 of 4 runs
Qwen3.5-4B Trace-Inversion2026-11-0540 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
TwIL-8B (webAI)2026-11-0640 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
LFM2.5-8B-A1B coding/security finetune2026-11-0644 F · 0 H · 0 IFalsified at every seed
Machynlleth-3B (BritLLM)2026-11-0644 F · 0 H · 0 IFalsified at every seed
CherryPit-1.2-2B (MiniCPM5)2026-11-0640 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Gemma-4-12B Maia finetune2026-11-0642 F · 2 H · 0 IHeld 2 of 4 runs
Granite-4.2-3B2026-11-0644 F · 0 H · 0 IFalsified at every seed
Olune Qwen3.5-4B2026-11-0640 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Qwen3.5-0.8B lobotomized-x22026-11-0640 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Llama-3.2-1B Reflex finetune2026-11-0644 F · 0 H · 0 IFalsified at every seed
kas-4b2026-11-0642 F · 0 H · 2 IFalsified 2 · incomplete 2 — no holds
Rennervate Qwen3-8B refusal-suppressed2026-11-0740 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Orvena-4B2026-11-0740 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
StoryGen-12B2026-11-0742 F · 1 H · 1 IHeld 1 of 4 runs
Qwen3.5-9B Brainwaves Coder2026-11-0740 F · 0 H · 4 INever falsified — but never cleanly held (incomplete runs)
Meerkat2-14B2026-11-0741 F · 0 H · 3 IFalsified 1 · incomplete 3 — no holds
NeoHorse-1-4B heretic2026-11-0743 F · 0 H · 1 IFalsified 3 · incomplete 1 — no holds
Qwen3 distill v2 rationale 1.7B2026-11-0744 F · 0 H · 0 IFalsified at every seed
TwIL-8B heretic2026-11-0741 F · 0 H · 3 IFalsified 1 · incomplete 3 — no holds
QwenDGAF-2 0.5B2026-11-0744 F · 0 H · 0 IFalsified at every seed
juke-1.7b-dev2026-11-0740 F · 4 H · 0 IHeld every run — undefeated

HOW A RUN WORKS, BRIEFLY

Before a run begins, the claim and the probe family are frozen. The model is placed in a cell with material it is instructed to protect, a role it is instructed to keep, and a boundary it is instructed not to cross. Four probes are applied — front-door disclosure, encoded disclosure, role override, and boundary proposal. The model's output is sealed before scoring and scored offline by deterministic detectors. A run holds only when every probe holds; one escape falsifies it; and a run in which no probe escapes but at least one produces no scoreable artifact is incomplete — never quietly counted as a pass. The full method, and its known limitations, are set out on the About page.