Skip to content
项目

openai/frontier-evals has 1,342 GitHub stars and 184 forks. GitHot measured +2 stars in 24 hours and +5 in 7 days. Last checked 2026-11-16 02:53 UTC. Last push 2026-10-12.

这些数字如何统计

实测星标增长

24小时
+2
7天
+5
15天
正在收集历史
1个月
正在收集历史

各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。

关于

OpenAI Frontier Evals

开发工具

README.md

原文

Frontier Evals

Code for evals measuring frontier model capabilities.

Usage

Requirements

We manage environments with uv. Install uv once, then run uv sync (or uv pip install -r ...) inside the project of interest to create its virtual environment from the checked-in uv.lock.

Running Evals

Each eval directory documents how to reproduce runs, configure models, and interpret results. Start with the suite README.md, then consult any scripts under scripts/ or runtime_*/ directories for orchestration details. When in doubt:

  1. cd into the eval directory.
  2. uv sync to install dependencies.
  3. Follow the local instructions in the README.md.

Contributing

Layout

.
├── pyproject.toml             # Shared tooling configuration (Ruff, Black, etc.)
└── project/
    ├── common/                # Shared libraries
    ├── evmbench/              # EVMBench eval
    ├── paperbench/            # PaperBench eval
    └── swelancer/             # SWE-Lancer eval

Each eval directory is its own isolated project with a README.md, pyproject.toml and uv.lock.

Development Workflow

  • Create or activate the environment for the project you are working on with uv. Example for PaperBench:
    • cd project/paperbench
    • uv sync
    • uv run pytest
  • Code style and linting use Ruff (with autofix profiles in pyproject.toml and project/common/tooling/ruff_autofix_minimal.toml) and Black. Run uv run ruff check --fix or use the provided Poe/make tasks where available.
  • Shared utilities live under project/common; changes there may affect multiple evals. Bump the relevant editable dependencies if you create new shared subpackages.