AgentRE Challenge / Season 1

Can You Build a Binary That Frontier AI Cannot Reverse Engineer?

A simple 33 days competition for original Linux x86-64 ELF binaries that meaningfully challenge frontier AI agents using static analysis.

Register Entry Submission Instructions Read Rules View Leaderboard
Competition Overview

Build a challenge, keep artifacts private, register with a public-safe PR.

Entrants keep binaries, checksums, metadata, and ground truth in private GitHub repositories. Any source code and build instructions requested from finalist or prize-eligible entries are provided privately and remain confidential. The public site shows only safe registration status and sanitized results.

Awards

First place receives $1,000 plus one month of the recipient's choice of ChatGPT Pro or Claude Max 20x. Second place receives one month of the recipient's choice of ChatGPT Pro or Claude Max 5x.

Target

Submissions must target Linux x86-64 ELF, remain under 10 MB, and be suitable for static reverse engineering.

Privacy

Public source release is not required. Private source, binaries, repository URLs, ground truth, evaluation notes, and full model transcripts remain confidential unless the entrant chooses to publish them.

Timeline

33 days from launch to winner announcement.

Calendar dates are set in the central challenge configuration. The Season 1 operating model is fixed around submission, validation, evaluation, and announcement windows.

Submission Workflow

Eight steps from private repo to evaluation.

  1. Build an original reverse-engineering challenge.
  2. Store it in your own private GitHub repository.
  3. Invite the AgentRE organizer agentrebench as a collaborator.
  4. Register with a public-safe pull request.
  5. Push the final tag agentre-season-1-final and register the resolved 40-character commit SHA.
  6. AgentRE validates and evaluates the binary.
  7. Frontier AI agents attempt to reverse engineer it.
  8. After validation and tie-breaks, AgentRE assigns the first- and second-place awards to the highest-ranked eligible submissions.
Register Entry Submission Instructions
Eligibility

Valid entries are original, reproducible, safe, and privately reviewable.

Required

  • Original work or work the entrant has the right to submit
  • Complete source code and reproducible build instructions provided privately if requested for finalist or prize verification
  • A verifiable intended behavior and private ground truth
  • Matching binary SHA-256 and final tag before the deadline
  • Organizer collaborator access through the end of judging

Not a win condition

  • Broken files, unsupported formats, or tool crashes
  • Pathological or intentionally excessive resource use whose primary purpose is infrastructure abuse
  • Undocumented runtime dependencies or missing files
  • Infrastructure abuse or unsafe payloads
  • Winning by hiding behavior outside the submitted commit
Strong Challenge Design

Reward real static reverse-engineering skill.

Season 1 focuses on meaningful static analysis difficulty rather than aggressive anti-analysis. A strong entry makes the model recover intent through evidence, not through guessing or infrastructure failure.

Reasoning depth

Indirect data flow, cross-function reasoning, state-machine reconstruction, and meaningful control-flow analysis.

Recoverable semantics

Sparse but recoverable clues, custom protocol recovery, algorithm identification, and compiler optimization effects.

Clear answer key

Private ground truth should explain important functions, key behaviors, traps, partially correct findings, and scoring expectations.

Judging

The hardest binary is the one the official panel understands least.

AgentRE scores how much required behavior each model correctly recovers. The primary ranking is lowest average official model correctness, reported as entry difficulty score: 100 - average official model correctness. The first-place entry must also be lowest-scoring on at least 4 of 6 official models.

Official panel

Season 1 uses the same six frontier-model runs represented on the public AgentRE leaderboard at competition opening. Only AgentRE-verified runs count toward rankings and awards.

Invalid failures do not count

Tool crashes, API errors, malformed inputs, missing dependencies, challenge-unrelated infrastructure abuse, and unrelated safety refusals are invalid runs. A model's failure to solve a valid difficult binary within the standard budget is a valid outcome.

Official Model Panel

Six frontier models, one standardized evaluation.

Allowed Tools

Models get the standard AgentRE static-analysis tools.

Official scoring uses a fixed AgentRE toolset from agentre-bench-tools:latest. Models may not generate and execute arbitrary Python or code in other programming languages. Allowing custom parsers, symbolic executors, deobfuscators, or other new tools during evaluation would make model comparisons unfair and difficult to reproduce.

Private Submissions

Registration keeps sensitive artifacts private.

Public during submissions

  • Entry ID after assignment
  • GitHub handle when permission is granted
  • Challenge title when permission is granted
  • Registration, validation, and evaluation status

Never public during submissions

  • Private repository URL
  • Binary, source code, or ground truth
  • Detailed challenge technique
  • Interim scores, interim model results, or full transcripts
Safety Requirements

Controlled, inert simulation only.

AgentRE may reject unsafe, copied, trivial, broken, unverifiable, or low-quality submissions. Security-relevant behavior must be inert, isolated, safe, clearly disclosed in private ground truth, and useful for reverse-engineering evaluation.

Do not submit real malware or unsafe automation.

Entries must not perform credential theft, credential collection, persistence intended for deployment, propagation, destructive payloads, ransomware behavior, external downloads, real command-and-control, exploitation of third-party systems, personal-data collection, or build scripts/workflows intended to compromise infrastructure.

Live Registration Summary

Public status, private artifacts.

This summary is loaded from public-safe registration JSON files merged through pull requests. No private repository URLs, binaries, source code, ground truth, private evaluation notes, or full model transcripts are stored in this public website repository.

0Registered entries
0Validated entries
0Evaluated entries
1Minimum accepted entry for prize
View Leaderboard
FAQ

Season 1 operating details.

Why are repositories private?

Private repositories prevent competitors and future models from seeing binaries, source code, challenge techniques, and ground truth during judging.

Will other competitors see my code?

No. AgentRE reviews submissions privately. Public releases require separate participant approval.

Will AgentRE publish my binary?

No automatic publication occurs. Public release of source or binaries requires separate participant permission.

Can I submit more than one entry?

Yes. One entrant or team may submit up to 3 entries and may win only one prize.

Can I update my repository?

Yes before the final deadline. AgentRE evaluates only the commit resolved from the final tag before the deadline.

Which commit will be evaluated?

The commit resolved from agentre-season-1-final, recorded as a full 40-character SHA in the public-safe registration PR.

Must I publish or submit source code?

No public source release is required. AgentRE may request source code and build instructions privately from finalist or prize-eligible entries to verify ownership, safety, reproducibility, intended behavior, and ground-truth accuracy. Those materials remain confidential unless you choose to publish them.

Can I use obfuscation?

Yes, when it supports meaningful static reverse-engineering difficulty and is disclosed in the private ground truth.

Can I submit malware?

No real malware. Controlled simulation may be accepted only when inert, isolated, safe, clearly disclosed, and useful for evaluation.

What are the awards?

First place receives $1,000 plus one month of the recipient's choice of ChatGPT Pro or Claude Max 20x. Second place receives one month of the recipient's choice of ChatGPT Pro or Claude Max 5x.

What happens if no entry qualifies?

If no entry passes validation, evaluation cannot produce a reliable result, or no entry satisfies the 4-of-6 first-place requirement, AgentRE may decline to issue either award.

Can AgentRE use my work commercially?

Submission grants only the limited permission needed to inspect, clone, rebuild, evaluate, store an internal evaluation copy, score, and announce results. Commercial use, training use, benchmark inclusion, dataset resale, or redistribution requires a separate explicit agreement.

Can my GitHub handle be hidden?

Yes. The registration JSON includes display flags for the public leaderboard. The pull request itself is public.

When are scores published?

Scores are published after judging and review, with award recipients announced on September 3, 2026.

What happens after Season 1?

Season 1 can be archived. Sanitized summaries and optional public repository links may be added only after participant approval.