openai/frontier-evals
openai/frontier-evals has 1,342 GitHub stars and 184 forks. GitHot measured +2 stars in 24 hours and +5 in 7 days. Last checked 2026-11-16 02:53 UTC. Last push 2026-10-12.
实测星标增长
- 24小时
- +2
- 7天
- +5
- 15天
- 正在收集历史
- 1个月
- 正在收集历史
各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。
关于
OpenAI Frontier Evals
开发工具README.md
原文Frontier Evals
Code for evals measuring frontier model capabilities.
- PaperBench: End-to-end replication of state-of-the-art AI papers. Paper | Blog
- SWE-Lancer: Real freelance software engineering tasks with end-to-end tests. Paper | Blog
- EVMbench: Smart contract security tasks. Paper | Blog
Usage
Requirements
We manage environments with uv. Install uv once, then run uv sync (or uv pip install -r ...) inside the project of interest to create its virtual environment from the checked-in uv.lock.
Running Evals
Each eval directory documents how to reproduce runs, configure models, and interpret results. Start with the suite README.md, then consult any scripts under scripts/ or runtime_*/ directories for orchestration details. When in doubt:
cdinto the eval directory.uv syncto install dependencies.- Follow the local instructions in the
README.md.
Contributing
Layout
.
├── pyproject.toml # Shared tooling configuration (Ruff, Black, etc.)
└── project/
├── common/ # Shared libraries
├── evmbench/ # EVMBench eval
├── paperbench/ # PaperBench eval
└── swelancer/ # SWE-Lancer eval
Each eval directory is its own isolated project with a README.md, pyproject.toml and uv.lock.
Development Workflow
- Create or activate the environment for the project you are working on with
uv. Example for PaperBench:cd project/paperbenchuv syncuv run pytest
- Code style and linting use Ruff (with autofix profiles in
pyproject.tomlandproject/common/tooling/ruff_autofix_minimal.toml) and Black. Runuv run ruff check --fixor use the provided Poe/maketasks where available. - Shared utilities live under
project/common; changes there may affect multiple evals. Bump the relevant editable dependencies if you create new shared subpackages.