Superforecasting: The Art and Science of Prediction

Name: Superforecasting: The Art and Science of Prediction
Author: Philip E. Tetlock & Dan Gardner

Philip E. Tetlock & Dan Gardner, 2015 — risk.

The empirical answer to whether geopolitical forecasting is possible at all. In the IARPA tournaments of 2011–2015, Tetlock’s Good Judgment Project put thousands of volunteers against dated, Brier-scored questions; the best of them reportedly outperformed professional analysts with access to classified information. The winning habits were unglamorous: granular probabilities, frequent small updates, decomposition of hard questions, and unsentimental post-mortems. This book is the operating manual for our Calibration Ledger — sealed claims, dated resolution, Brier scoring, misses kept on the record.

Why the engine keeps this on the shelf

This is the rulebook the sealed ledger enforces — sealed probability, named adjudicator, fixed resolution date, Brier score published whether or not it flatters the engine.

The record

Crown published it in September 2015, reporting on IARPA's Aggregative Contingent Estimation tournament, which ran 2011–2015 across roughly 500 questions and more than a million individual forecasts.
The Good Judgment Project, led by Philip Tetlock and Barbara Mellers at the University of Pennsylvania, won the tournament outright; by its fourth year the aggregated GJP forecasts were about 50% more accurate than IARPA's own public control group.
Its best forecasters were reported to beat intelligence analysts working with classified material by roughly 30% on the same dated questions.
The book answers Tetlock's own earlier finding: Expert Political Judgment (Princeton, 2005) collected some 28,000 forecasts from 284 experts over two decades and found the average expert barely better than chance.

Marked passages

Foresight isn't a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs. These habits of thought can be learned and cultivated by any intelligent, thoughtful, determined person.

The premise of the Observer Protocol: forecasting skill is trainable and measurable, so reputation should be earned through scored track records rather than credentials or volume.

The scoring discipline, summarized: a forecast without a date, a probability, and a resolution criterion is not a forecast — it is a mood. Only claims that can be scored can be trusted, and only scored forecasters can improve.

Why the ledger was sealed on 2026-07-03 with all entries marked pending, and why the first Brier scores arrive in July 2027 and not before. No track record is claimed until one exists.

The core claims

Forecasting accuracy is a trainable skill, and the habits that produce it — decomposing a question, starting from base rates, updating in small frequent increments — are teachable rather than innate.
A claim that cannot be scored is not a forecast, so the discipline lives in the resolution criterion and the date rather than in the persuasiveness of the argument.

Then and now

The human-versus-machine rerun of his tournament is live: as of May 2026, professional forecasters still led the bots head-to-head on Metaculus's benchmark, having taken every quarterly round across roughly eighteen months. Source: Metaculus AI Forecasting Benchmark / FutureEval leaderboard, May 2026

In the Spring 2026 Metaculus AI Benchmark Cup the strongest in-house bot placed 33rd against 1,130 human forecasters — the top ~3%, and still not the top. Source: Metaculus, AI Forecasting Benchmark Spring 2026 Cup results

This text points at

TALEB

The shelf exists because the engine reads it. See the Core, the projections, the sealed ledger, or all 81 texts.