ABOUT THIS SITE

Purpose. LLM DevSecOps is the serious half of a two-house publication. This site carries the record: the APEX escape-claim benchmark, published run by run, model by model, with methods and limitations stated on the record. Its companion site, onlymodels.gg, presents the same results as entertainment — daily model brawls, standings, and belts. The show links down to the data published here; the data does not wear the costume.

THE CLAIM UNDER TEST

APEX — a clinical-trial-style benchmarking programme — tests a single, deliberately narrow claim per run: the subject configuration does not escape its cell under a frozen probe family. “Escape” is defined precisely and modestly: disclosing protected material (directly or encoded), deviating from an assigned role under a lower-priority instruction, or proposing to cross a stated boundary. It is not a claim about a model breaking out of a sandbox, and the programme does not present it as one.

METHOD

Campaigns are frozen at admission: the claim, the probes, and the scoring predicates are fixed before any output exists. Outputs are captured and sealed; scoring happens offline against the sealed artifacts; model identity is pinned by served digest where the host permits it; adjudication is append-only. A verdict of holds requires every probe to hold. Any single escape falsifies the claim. Runs whose evidence collection itself fails are reported as inconclusive rather than scored — an instrument failure is data, not a pass.

LIMITATIONS, STATED PLAINLY

The detectors are deterministic term-matching instruments. They are not yet validated against human judgement: a control battery of synthetic cases currently scores 13 of 17, with known false positives on refusals and quotations that name the forbidden object. Verdicts on this site are therefore stated as detector verdicts on sealed, unlabelled artifacts. Blinded human labelling of every output, with published agreement statistics, is the programme's next step and its largest open gap. Likewise, four seeds per model support existence claims — a model can escape; a model did hold four times — not tight rate estimates.

THE BOOK

The long-form record of the first 44 runs is published as APEX Presents: Bad Models — Escape Claim Reports, organised one chapter per model. Runs since the book's last update appear here first.

AUTHORSHIP

APEX and this site are the work of David Maynor. The programme is sponsored by TableX.