About the Foundation
A neutral home for measuring AI in finance
The Alpha Bench Foundation maintains Alpha Bench, an open benchmark for AI in finance. We exist to help every language model, agent harness and vertical agent see where it falls short and how to improve — so the financial help people receive keeps getting better.
Our purpose
Measure honestly
Point-in-time data, realistic frictions, many market regimes and enough trials to tell skill from luck. A result should mean exactly what it says.
Help systems improve
We evaluate in order to improve. Results are broken down until they point to something a builder can change, then tested again on tasks the system has never seen.
Serve the people who rely on them
The goal is not a higher score. It is financial help people can rely on: clear about risk, suited to their situation and honest about what it does not know.
Stay neutral
We favor no model, lab or approach. Every system is evaluated under the same conditions and judged by the same standard.
Our work
What the Foundation does
Alpha Bench is still being built. At every stage, these are the responsibilities the Foundation exists to carry.
- 01
Maintain the benchmark
Design and refresh tasks across twelve strategy families and four tracks, retire tasks that have leaked, and keep every version reproducible.
- 02
Run evaluations
Evaluate models, harnesses and vertical agents under identical, controlled conditions — and return a diagnosis with every score.
- 03
Publish methodology and reports
Release the methodology, evaluation reports and a versioned changelog, so anyone can see how a number was produced and what has changed.
- 04
Convene practitioners and researchers
Bring finance practitioners, AI researchers and builders together to decide what is worth measuring, and how to measure it well.
- 05
Steward datasets and licenses
Curate point-in-time datasets, honor the licenses they come with and make clear what participants may use and share.
Principles
What we value and how we act
Measuring AI in finance means making judgment calls every day: what to test, what counts as a good answer, what is ready to publish. These principles guide those calls, and how we work with everyone who takes part.
Be right before being early.
We would rather publish later than publish something we cannot stand behind. Methods, tasks and results are checked before release, and corrected in the open when we get them wrong.
Show the work.
A number without its reasoning helps no one improve. We explain how every score is produced, and we ask the systems we evaluate to show their evidence and reasoning too.
Put the user first.
Behind every task is someone who may one day rely on the answer. We measure what matters to them — suitability, clarity, honest disclosure of risk — not only what is easy to score.
Stay independent.
We follow the evidence, not the preferences of any vendor, funder or participant — including when the results are inconvenient.
Measure the hard parts.
Costs, liquidity, regime shifts, uncertainty and knowing when to say no are hard to evaluate. They are also where real money is lost, so we measure them rather than leave them out.
Keep learning.
Markets change, and so do the systems we test. When the evidence shows a task or method falls short, we revise it and record the change in a public changelog.
Our mission
We measure so that systems can improve. Our mission is to help every model, harness and vertical agent become measurably better at finance — and to make sure that progress reaches the people who turn to them for help with their money.
Governance
How we stay neutral
Alpha Bench is only useful if the people it measures, and the people who rely on it, can trust it. These are the commitments we hold ourselves to — and ask to be held to.
- 01
Open methodology, public changelog
We commit to publishing how every task is built and scored, and to recording each change to tasks, metrics and rules in a versioned, public changelog.
- 02
Funding never buys results
We commit to keeping funding and evaluation apart. No funder, sponsor or partner can buy placement, influence a score or decide which results are published.
- 03
Conflicts of interest disclosed
We commit to disclosing relationships between the Foundation, its contributors and the systems under test, and to keeping anyone with a conflict out of the decisions it touches.
- 04
Independent expert review
We commit to having methods and task sets reviewed by independent finance practitioners and researchers before they are used for published results.
- 05
Held-out tasks stay held out
We commit to never showing held-out or live tasks to participants, and to retiring any task we have reason to believe has leaked.
- 06
Results with uncertainty and caveats
We commit to publishing every result with its uncertainty, the conditions it was measured under and the limits of what it shows.
The Foundation’s governance is still taking shape. We will publish more detail as it does.
Ask us about governanceHelp us measure what matters.
Alpha Bench is being built in the open. If you build AI systems, work in finance or study how to evaluate either, we would like to hear from you.