AI Control Hackathon 2026 retrospective
In short
From June 11 to June 18, ~50 people joined us for the AI Control Hackathon 2026, a one-week event on effectively using an untrusted AI system while minimizing the harm it can cause, even when it actively tries to subvert the controls around it. Eight teams formed across blue (defense), red (attack), and white (arena) tracks.
Horizon Omega and HackOS organized it. The event was hybrid: at Ω Labs, the Montréal AI safety coworking space, and online.
It was sponsored by Redwood Research, NVIDIA, Rootly, and Rubric Labs.
AI Control and the hackathon setup
AI Control is a field focused on getting useful work from AI systems capable enough that we can’t rule out their trying to subvert safeguards. The setup is an adversarial game: a Blue Team designs protocols (typically a weaker, trusted model that scores each action’s suspiciousness) while a Red Team uses a more capable, untrusted model to accomplish harmful “side tasks” while passing as honest “main task” work. Protocols are stress-tested with control evaluations that pair honest and attack runs and score each protocol on two axes: safety and usefulness. The field began with Greenblatt & Shlegeris’ AI Control: Improving Safety Despite Intentional Subversion and was scaled up by LinuxArena, now the largest software engineering AI Control setting.
For the hackathon, participants picked between the following sides:
Blue Team: building defenses, with Track A writing a small trusted monitor that scores each action’s suspiciousness, calibrated to flag about 1% of honest actions; Track B designing full protocols (defer to the trusted model when suspicious, gate permissions) and measuring their safety-usefulness tradeoff
Red Team: stress-testing the aforementioned defences, with Track C focusing on hand-crafting a sabotage trajectory that reads as ordinary work to the monitors, and Track D breaking a shipped protocol and proposing a fix.
White Team (Track E): meaningfully extend the LinuxArena benchmark, e.g. with new environments and tasks.
Winning projects
1st place: Team IIT (Mazin Khider, Iqtedar Uddin), for making the monitor’s alarm threshold statistically defensible.
When is a “1% audit budget” actually 1%? They show that an observed sub-1% false-positive rate doesn’t statistically prove the true rate is under 1% at these sample sizes, and they apply confidence bounds to find which thresholds survive scrutiny. Their monitor, zmean3, ensembles three existing monitors by normalizing each to honest-run data only and averaging. Finer score granularity preserves more safety under certification: zmean3 exposes ~103 distinct honest score levels versus the baseline’s 15, so it can sit closer to the budget boundary. It beats the baseline at 1%. (Repository, Report)
2nd place: Team Sea Blue (Thomas Gaviard, Mathias Pacheco Lemina), for a cheap two-stage monitor, plus an interactive design that gets agents to confess.
Their two_layer monitor runs a cheap first pass that also marks a confidence bit, and escalates only the ~8% of uncertain actions to a stronger second model. It beats the baseline at every untrusted-model capability level. A second design, ask, lets the trusted monitor question the untrusted agent directly: it underperforms on average (agent awareness inflates honest scores too), but it produced a striking result: in 46 of 47 high-score questioned actions, the untrusted agent confessed its hidden objective. (Repository, Report)
3rd place: Team Alpha Nova (Laxman KC, Athif Shaffy, Cheick Ismael Maiga), for matching baseline safety at a quarter of the model calls, with rigorous ablations.
AgentTracer is a dual monitor: a per-action LLM scorer that contrasts each action against the main task, plus a single whole-trajectory pass that catches the dispersed, multi-step sabotage a per-action view structurally misses. It matches the baseline’s safety at 1% within overlapping confidence intervals, but at roughly a quarter of the model calls per trajectory. Their ablations are rigorous: ensembling more monitors diluted results, an agentic triage-then-investigate variant sharpened single cases but was noisier, and on a strong model the baseline nearly catches everything in the hardest dataset. The conclusion: passive monitoring plateaus; the frontier is the subtle, dispersed sabotage that hides across individually benign steps. (Repository, Report)
Additionally, our judges found that the following teams stood out within their respective tracks:
Track A (monitor building) winner: Sea Blue
Track C (attack crafting) winner: pamplemousse (Ali Meguellati, Maxim Qu, Cheela Zhu)
Track E (new control settings) winner: IIT
NVIDIA Brev winner: Alpha Nova
Acknowledgements and conclusion
We’re thankful towards Redwood Research for providing infrastructure and generous API credits to each team, 1100$ in cash prizes, and for providing judging capacity; to NVIDIA for providing GPU credits for participants and the 400$ Brev credits prize; and to Rootly AI Labs and Rubric Labs who generously covered the food and drinks of the opening ceremony and demo night.
And last of all, we’re glad to have been able to organize this hackathon in collaboration with HackOS, and we’re thankful to the participants, whether in person or online, and to everyone who made time in their schedules, for your help in advancing AI safety as a research field.
This was our first end-to-end hackathon and we learned greatly from it. Thanks all for participating!
