// HACKER NEWS — CYBERSECURITY
What happens when you analyze your favorite college football team like the CIA?
A couple weekends ago, the Illinois football team lost to Duke at home,
31–27. My Hinsley model immediately became much less optimistic about Illinois making
the College Football Playoff.
But it became more optimistic about the offensive line and our new
quarterback.
That sounds contradictory, but it's exactly what I wanted to happen.
For the past 15 years, we've worked with people whose job is to make
judgments about uncertain futures: intelligence and government analysts,
foreign-policy researchers, investors, and corporate strategists. This year I
decided to try an experiment. I took the methodology we've developed for
that kind of work and applied it to something considerably less consequential:
assessing the fortunes of the University of Illinois football team.
To understand why, it helps to think about what an intelligence analyst
actually does.
During the Cuban Missile Crisis, American intelligence analysts were trying
to understand what the Soviet Union was doing in Cuba. They had a growing
collection of evidence, but the difficult part was deciding what it meant.
Analysts had to consider competing explanations, identify the observations that
distinguished one from another, and revise their assessments as new evidence
arrived. Eventually, U-2 photography provided much stronger evidence that the
Soviets were installing nuclear missiles.
The stakes are obviously rather different, but the analytical problem is
surprisingly general. Usually there isn't one fact that gives you the
answer. There are several possible futures, a huge amount of imperfect
information, and a smaller number of things that actually help distinguish
among them. The analyst's job is to impose structure on all of this without
becoming more certain than the evidence warrants.
You find versions of this problem everywhere. A foreign-policy analyst might
be trying to understand whether a conflict will escalate. An investment analyst
might be thinking about how geopolitics, regulation, or a new technology will
affect an asset over the next decade. A government analyst might be assessing
how another country will respond to a policy change. The useful question
isn't simply, "What do I think will happen?" It's: What are
the plausible ways this could turn out? What would have to be true for each of
them? What should I be watching? And what new evidence would cause me to change
my mind?
It's not broadly known, but for more than a decade, Cultivate ran a
prediction market for the U.S. Intelligence Community, giving analysts a way to
make and aggregate probabilistic forecasts about geopolitical and
national-security events. More recently, our work has expanded beyond
forecasting individual questions into the broader analytical process around
them.
That's what led us to develop Continuous
Probabilistic Foresight, or CPF, the methodology at the heart of Hinsley, our AI/human hybrid analysis platform. CPF
starts with a strategic question and maps the range of plausible outcomes as
scenarios. It decomposes the problem into the drivers and indicators that would
make those scenarios more or less likely, makes assumptions explicit, and turns
important uncertainties into resolvable forecasting questions. As new evidence
arrives, those forecasts and the larger assessment can change with it.