AgentRE Challenge / Register

Register an entry

Register With a Public-Safe Pull Request

Your actual submission stays in your private GitHub repository. The pull request only registers public-safe metadata so AgentRE can track status on the leaderboard.

PR RegistrationOpen
PublicOne safe JSON registration file
PrivateBinary, ground truth, and verification materials stay out
agentre-challenge-entryRequired private repo name
StaticLeaderboard updates after merge

Open a PR that adds only public-safe metadata.

  1. Create your private submission repository named agentre-challenge-entry.
  2. Add agentrebench as a collaborator.
  3. Prepare the required private binary, checksums, metadata, and ground truth. If AgentRE requests source code and build instructions for finalist or prize verification, provide them only through the private repository.
  4. Fork the public AgentRE-Bench repository.
  5. Copy docs/challenge/registrations/template.json to a new filename in docs/challenge/registrations/.
  6. Fill in the public-safe registration metadata, including the final tag, resolved commit SHA, checksum, and eligibility confirmations.
  7. Add your filename to docs/challenge/registrations/index.json.
  8. Open a pull request using the challenge registration template.

Use this shape in the public PR.

The file is public. It intentionally does not include a private repository URL. AgentRE infers the private repository from your GitHub handle and the required repository name agentre-challenge-entry.

{
  "entry_id": "PENDING",
  "github_handle": "@example-user",
  "challenge_title": "Withheld until judging",
  "public_handle": true,
  "public_title": false,
  "private_repository_name": "agentre-challenge-entry",
  "entrant_submission_number": 1,
  "final_tag": "agentre-season-1-final",
  "final_commit_sha": "RESOLVED_FULL_40_CHARACTER_COMMIT_SHA",
  "binary_sha256": "SHA256_OF_BINARY",
  "registration_status": "Registered",
  "validation_status": "Pending",
  "evaluation_status": "Pending",
  "repo_private_confirmed": true,
  "organizer_invited": true,
  "required_files_confirmed": true,
  "rules_read": true,
  "age_18_or_older_confirmed": true,
  "us_eligibility_confirmed": true,
  "free_entry_confirmed": true,
  "designated_prize_recipient_confirmed": true,
  "original_work_or_rights_confirmed": true,
  "no_private_urls_or_artifacts_in_pr": true
}

Add your registration file to the static manifest.

The leaderboard loads every file listed in docs/challenge/registrations/index.json. Add only your filename, not a private URL.

{
  "files": [
    "example-user-example-challenge.json"
  ]
}

The registration PR is public. Keep private material out.

Allowed in the PR

  • GitHub handle
  • Public challenge title or "Withheld until judging"
  • Final tag and resolved full 40-character commit SHA
  • Binary SHA-256
  • Display permissions, ownership confirmation, and eligibility confirmations

Never include in the PR

  • Private repository URL
  • Binary attachments or downloads
  • Source code
  • Ground truth
  • Secrets, tokens, credentials, or private keys
  • Private evaluation notes or model transcripts