Len’s Focus: Predictions, Past & Present

Written by Len Asimow, head of the Department of Mathematics at Robert Morris University and founding Director of the Actuarial Science Program, currently designated a Center of Actuarial Excellence by the SOA:

OLYMPUS DIGITAL CAMERA

He is a co-author (with Mark Maxwell) of Probability and Statistics with Applications (ACTEX Academic Series, 2010). His blog posts will come from ‘Actualties’ which is RMU’s actuarial program newsletter.

“I never make predictions, especially about the future.” – Yogi Berra

When prospective students ask about the difference between accountants and actuaries we tell them, “Actuaries predict the future, accountants predict the past.”  If he had to choose, Yogi Berra would probably be more comfortable as an accountant, predicting the past.  Fortunately he found his real calling on the baseball field.  In defense of accountants, while predicting the past is pretty easy, interpreting it is much harder.  And accountants, for the most part, do a pretty good job of that.  But predicting the future is where the real action is.

Actuaries are paid to predict and quantify future financial risks.  As Yogi intuited, the chief risk to the actuaries themselves lies in getting it wrong.  In most cases this is simply because of the vicissitudes of nature and the inherent randomness of the future.  Generally, this merely results in embarrassment and repetitional damage; however it is not unprecedented for actuaries to be sued for fraud and/or negligence.  As the saying goes, “… to over-reserve has been said to be bad practice, to under-reserve is said to be worse, to be exactly right is said to be a miracle.”

Of course missing forecasts isn’t just a problem for actuaries.  History is replete with famously wrong predictions.  For example, we are still here, despite the widely circulated prophesies for the apocalypse on 12/21/2012 based on the Mayan calendar. That may be a little embarrassing for the prophets, but hey, it’s not the end of the world.

So how do actuaries mitigate the risks of making bad predictions?  First, in making calculations (about reserves, pricing, etc.) actuaries are expected to perform their duties with “the degree of skill and competence reasonably expected.”  Second, actuaries must adhere to “generally accepted actuarial principles.”  Third, make sure the malpractice insurance is paid up.

So far we have focused on the consequences of making erroneous predictions, that is, predicting something that does NOT happen.  In statistical parlance this might be considered a Type I error.  What about Type II errors, that is, failing to predict something that DOES happen?  It seems somewhat counterintuitive that a failure to predict (the default position of Yogi Berra and millions of others) could result in dire consequences.

But, consider the plights of six scientists (plus a politician) who were convicted of manslaughter by an Italian court for failing to predict a 2009 earthquake that led to more than 300 deaths and devastated the region in central Italy around the city of L’Aquila.  The impact of this verdict on the scientific community has been, well, earthshaking.  After all, no one has yet determined how to predict the precise time and place of earthquakes, so this might be the most sensational injustice to come out of Italy since the Galileo heresy trial in 1633.  Thousands of scientists in Italy and around the world have signed petitions expressing outrage and demands to free the Italian six.  (Everyone agrees the politician can rot in jail.)

From an actuarial standpoint earthquakes, along with other cataclysmic natural disasters, occur randomly and sporadically (think Poisson!), conforming to no discernible pattern.  So the court finding seems to be perverse to say the least.  On the other hand, Prob/Stat students might appreciate an attempt to say something about the frequency of occurrences of future earthquakes.  So here goes, at the risk of exposure to future legal proceedings.

The U.S. Geological Survey maintains databases on earthquakes.  It turns out that during the 40 years preceding the year 1973 the number (worldwide) of major quakes (on the order of magnitude 8 or more on the Richter scale) averaged about 0.875 events per year.  Let’s imagine that as forecasters living in the year 1972 we use the Poisson distribution and the rate of 0.875 to predict the frequencies of earthquakes 40 years into the future.  We then put our results in a time capsule to be opened at the end of 2012.

We would have predicted no major quakes for 42% of the years, one major quake in 36% of the years, and two or more major quakes in the remaining 22% of the years.

If we were a global earthquake insurance company this would give us a start on reserving for the really big ones.  Since 2012 is now behind us let’s open our time capsule and see how we did.

The case of the guilty Italian scientists has its comedic side, but there are also some serious considerations that help to explain the verdict.  A popular recent book entitled The Signal and the Noise, by Nate Silver delves deeply into the follies and pitfalls for those who (like actuaries) earn their daily bread by making predictions.  An entire chapter is devoted to the earthquake problem, along with chapters on baseball, economics, politics, the weather, and many other areas in which the foolhardy make predictions for a living.

Silver cites the startling conclusion of a 15 year study of expert predictions in all fields.  “The experts in this survey – regardless of their occupation, experience, or subfield – had done barely any better than random chance, and they had done worse than even rudimentary statistical methods at predicting future political events.  They were grossly overconfident and terrible at calculating probabilities:  about 15% of events that they claimed had no chance of occurring in fact happened, while about 25% of those that they said were absolutely sure things in fact failed to occur. It didn’t matter whether the experts were making predictions about economics, domestic politics, or international affairs; their judgment was equally bad across the board.”

With that warning in mind, here is my carefully hedged prediction for our students:  You are bound to be both elated and disappointed, but also amazed, dazzled and filled with wonder as your future lives unfold.