Skip to content
Alpha Bench
中文

Participate

Help build Alpha Bench

Alpha Bench is being built in the open. A benchmark is only as good as the questions it asks, the people who judge the answers and the systems willing to be measured. You can take part by submitting a system, contributing expertise or supporting the work.

Submit a systemContribute expertise

Submit a system

Evaluate first, then decide what to publish

For model labs, harness builders and vertical agent teams. Each kind of system is evaluated the way it is used: models inside a fixed reference harness, harnesses with fixed models, vertical agents end to end.

How systems are evaluated

  1. Register interest

    Email us with what you would like to evaluate — a model, a harness or a vertical agent — and who we should work with on your side.

  2. Integrate

    Models connect through the reference harness; harnesses and vertical agents implement the agent interface specification. Either way, every system works from the same tasks and the same point-in-time data.

  3. Run evaluations

    Your system runs versioned task sets across the four tracks: Analysis, Simulation, Live and Client. You receive a private diagnostic report with outcome and process scores by strategy family and track, and the failure modes found.

  4. Choose whether to publish

    From v1.0

    Reports stay private. From v1.0, you can choose to add your results to the public leaderboard. Nothing is published unless you decide to publish it.

Before you start

What you will need

  • A stable way to call your system: an API endpoint for a model, or an implementation of the agent interface specification for a harness or agent.
  • A version identifier for every submission, so each run can be reproduced exactly.
  • A list of the tools and data sources your system uses, and a way to restrict them — point-in-time tasks may only see what was knowable at the time.
  • A technical contact for integration questions.

Opening in stages

Where things stand

Submissions will open in stages. We plan to begin with a small group of early participants while the reference harness and the agent interface specification are finalized, and to widen access from there. We do not have dates to share yet.

Register interest

Contribute expertise

Help define what good looks like

Alpha Bench needs people who know markets and people who know measurement. You can contribute in Chinese or English.

Explore the twelve strategy families
Tasks

Task authors

Portfolio managers, analysts, traders and strategists with hands-on experience in any of the twelve strategy families.

What you would do

  • Draft point-in-time tasks drawn from real investment decisions
  • Write down what a strong answer contains — and what a weak one gets wrong
  • Check that every task can be answered with what was knowable at the time
Offer to write tasks
Rubrics

Reviewers and raters

Investment professionals, financial advisers and compliance specialists who can judge reasoning, not just results.

What you would do

  • Rate answers against the process rubric: evidence, reasoning, risk discipline, constraints, costs and calibrated uncertainty
  • Judge Client-track conversations for suitability, clarity and honest risk disclosure
  • Take part in calibration, so ratings stay consistent across reviewers and over time
Offer to review
Methodology

Researchers

Academic and industry researchers working on evaluation, statistics, finance or machine learning.

What you would do

  • Review the metrics, the scoring and how results are aggregated
  • Strengthen the statistics: enough trials, and the right tests, to separate skill from luck
  • Probe contamination: held-out sets, task rotation and leak detection
Discuss the methodology
Data

Data partners

Holders of point-in-time datasets: prices, fundamentals, filings, news, and alternative and macro data.

What you would do

  • Provide point-in-time history for research tasks and market replays
  • Help verify timestamps, revisions and survivorship
  • Agree access terms that keep every evaluation run reproducible
Discuss data access

Not sure where you fit? Write to us and tell us what you know.

Support the work

Help sustain an independent benchmark

Running an open benchmark well takes sustained funding, a great deal of compute and access to good data. Each kind of support helps — on one condition: it never shapes what we measure or what we find.

How the Foundation stays neutral

  • Funding

    Financial support for task authoring, expert review, data and the people who maintain the benchmark.

  • Compute

    Inference and compute credits to run evaluations at scale, with enough repeated trials to separate skill from luck.

  • Data access

    Point-in-time market, fundamental and alternative data, licensed for use in evaluation.

Our guardrails

Support can pay for the work. It cannot buy a result.

Discuss support

  1. Support never influences methodology, task selection, scoring or results.

  2. Supporters are disclosed publicly, together with the kind of support they provide.

  3. Supporters who also build AI systems are evaluated on exactly the same terms as everyone else.

  4. We decline support that comes with conditions on what we measure or publish.

FAQ

Questions about taking part

Some of this is still being decided. Where that is the case, we say so.

Ask us something else
When will submissions open?
Submissions will open in stages, and we are not giving dates yet. If you register interest, we will let you know when the next stage opens.
Does it cost anything to take part?
Registering interest costs nothing. How the cost of running evaluations — mostly compute — will be covered is still being decided. We will set it out plainly before anyone is asked to commit.
Can we evaluate privately, without publishing?
Yes. Every evaluation produces a private diagnostic report. There is no public leaderboard before v1.0, and from v1.0 publishing is your choice: results stay private unless you decide otherwise.
How are submissions kept confidential?
The principle is simple: your system’s outputs, reports and integration details are used to evaluate it and for nothing else, and are not shared with other participants or with supporters. Formal confidentiality terms are being drafted and will be in place before evaluations begin.
Can closed-weight models take part?
Yes. Alpha Bench evaluates systems through their interfaces — an API for a model, the agent interface specification for a harness or agent — so it does not need access to weights or source code. Open- and closed-weight systems are evaluated the same way.
Which markets and asset classes are covered?
The twelve strategy families span equities, government and corporate bonds, credit, currencies, commodities, futures and options. Which markets each task set draws on is being settled family by family, and will be documented with each version of the benchmark.
Can I take part in Chinese or English?
Either. You can write to us, author tasks and review in Chinese or English — whichever lets you be most precise.

Contact

Write to us. Every way of taking part begins with a short email.

Email

hello@alphabench.example

What to include

Who you are
Your name, your organization if you have one, and your role.
How you want to take part
Submitting a system, contributing expertise, supporting the work — or simply a question.
What you work on
For a system: whether it is a model, a harness or a vertical agent. For expertise: your strategy families or research areas.
Your language
Chinese or English, whichever you prefer.

Please leave confidential details out of a first message. If they are needed, we will agree how to share them.