8  Overconfidence

8.1 Ten questions

For each of the following questions, provide an upper bound and a lower bound that forms a 90% confidence interval for your answer.

In other words, there should be only a 5% chance that the true answer is below your lower bound (95% chance it is above) and a 5% chance that the true answer is above your upper bound (95% chance it is below). Do not research the answer. If you have very little concept of the answer, adjust your confidence intervals accordingly.

As an example, suppose I was asked the height of Mount Everest. I might guess a lower bound of 8500m, and an upper bound of 9000m. I should choose this bound so that I believe the true height has a 90% probability of being within those two bounds.

  1. What is the average orbital difference of the moon from the earth (in kilometres)?

  2. How many academy award nominations did Walt Disney receive?

  3. What age was Mahatma Gandhi when he died?

  4. What is the speed of light (in kilometres per second)?

  5. How many people is the certified maximum capacity of the Airbus A38?

  6. What is the world record for the women’s 100m sprint (in seconds)?

  7. How much tax revenue did the Australian (Commonwealth) Government collect in 2018-19 (in billions of dollars)?

  8. How many student enrolments did UTS have in 2019?

  9. How many meters high is Australia’s tallest building?

  10. How many test matches did Don Bradman play?

8.2 Above or below average

For each activity, rate whether you are above or below average?

  1. Driving ability
  2. Drawing ability
  3. Intelligence
  4. Ability to get along with others
  5. Juggling

8.3 Disambiguating overconfidence

De Bondt and Thaler (1995) wrote “Perhaps the most robust finding in the psychology of judgment and choice is that people are overconfident.” Take the following examples:

  • When asked to estimate the length of the Nile by providing a range the respondent is 90% sure contains the correct answer, the estimate typically contains the correct answer only 50% of the time.
  • PGA golfers typically believe they sink around 75% of 6 foot putts – some even believe they sink as many as 85% – when the average is closer to 55%.
  • 93% of American drivers rate themselves as better than average. 25% of high school seniors believe they are in the top 1% in ability to get along with others.

There are many similar examples, all making the case that people are generally overconfident.

But despite each being labelled as overconfidence, note that these examples are actually three different phenomena.

Overprecision is the tendency to believe that our predictions or estimates are more accurate than they actually are. The typical study seeking to show overprecision asks for someone to give confidence ranges for their estimates, such as estimating the length of the Nile. The questions asking for confidence intervals above were designed to test your precision.

Overestimation is the belief that we can perform at a level beyond that which we realistically can. The evidence here is mixed. When attempting a difficult task such as a six foot putt, we typically overestimate. But on easy tasks, the opposite is often the case – we tend underestimate our performance. Whether over or underestimation occurs depends upon the domain.

Overplacement is the erroneous relative judgement that we are better than others. Obviously, we cannot all be better than average. But this relative judgement, like overestimation, tends to vary with task difficulty. For easy tasks, such as driving a car, we overplace and consider ourselves better than most. But ask people where they rate for a skill such as drawing, and most people will rate themselves as below average. People don’t suffer from pervasive overplacement. Whether they overplace depends on what the situation is. The questions rating your relative skill above were a test of overplacement.

You might note that we tend to both underestimate and overplace our performance on easy tasks. We can also overestimate but underplace our performance on difficult tasks.

So are we both underconfident and overconfident at the same time? The blanket term of overconfidence does little justice to what is actually occurring.

The conflation of these different effects under the umbrella of overconfidence often plays out in stories of how overconfidence (rarely assessed before the fact) led to someone’s fall. For instance, evidence that people tend to believe they are better drivers than average (overplacement) is not evidence that overconfidence led someone to pursue a disastrous corporate merger (overestimation).

For more on disambiguating overconfidence, listen to Don Moore with Julia Galef (n.d.) on Rationally Speaking.

8.4 Overconfidence - a model

Don Moore and Paul Healy’s (2008) explanation for this mix of over and underconfidence on tasks of varying difficulty is that, after performing a task, we have imperfect information about our own performance. We have even less perfect information about that of others.

The result of the imperfect information is that people’s post-task assessments of their performance is regressive. That is, it tends to revert to the typical level of performance. Their perception of other’s performance is even more regressive.

When performance is high, the regressive assessment leads people to underestimate their own performance, but underestimate others even more so. Hence, they overplace themselves. For tasks with low performance, they overestimate their performance, but overestimate others even more. Hence they underplace for hard tasks.

As an illustration, consider an easy task, such as a test in an easy university subject. Suppose you know that the average mark in past years for this subject was 70%, and you expect to get the same result.

You then take the test, and think you might have done really well. “Did I just get everything right on that test? I could have got as high as 90%.”

However, you can’t be sure that your impression is correct. There might have been a trick question. So it makes sense to update your belief to a score of more than 70%, but perhaps not all the way to 90%. Let’s say the new estimate of our performance is 80%.

To the extent the new information is accurate (your score actually was 90%), you would be underestimating your performance.

But what of others? Before the test, you assumed they would get the same 70% as yourself. But you are now told that you scored 90%. Should you update the estimate of the other’s performance?

That depends on what you attribute your higher score to. Partly, it was easier than you expected, so it would have been easy for them. But it must also have been partly due to some idiosyncratic feature of your performance on the day. Maybe you blitzed it?

The result is that you update their estimated performance up from 70%, but not all the way to the 90% that you scored.

The result of this adjustment is that your believe your performance is higher than others, a feature of overplacement.

8.5 Overconfidence - firm entry

Most new businesses fail within a few years. For example, one study of US manufacturers found over 60% of entrants had exited within five years and almost 80% within 10 years.

Camerer and Lovallo (1999) ran an experiment to test whether business failure may be due to optimism about their relative skill.

The lab experiment involved a set of markets in which market entrants were paid a set amount according to their rank within the market. Those ranked within the “market capacity” would share a payment of $50. Those beyond the market capacity would be penalised $10. Accordingly, if there are 5 entrants above market capacity, the expected payoff of all entrants is zero. More than that and it is negative.

The rank in the market was determined by either luck, through a random draw, or a test of skill involving logic puzzles or trivia questions about sports or current events.

In each round of the experiment, the market capacity was announced to the players, along with whether the payoffs in the market were based on luck or skill. The participants were then asked to forecast the expected number of entrants (for which they earn a payment if correct) and decide simultaneously and without communicating whether to enter into the market. Subjects were then told how many participants had entered.

After all of the rounds, students solved puzzles or took the trivia quiz to determine their skill rank.

The results of the experiment showed that more participants entered the market when the ranking was based on skill than if based on random draw. This indicates a belief that their skill level will rank them higher than a random draw: they are above average.

An interesting element to this experiment was that for some of the markets the participants were recruited by being asked if they would like to volunteer for an experiment in which performance would depend on their performance on sports or current event trivia questions. Hence the pool in those markets would be stronger than typical.

In those markets with self-selected participants, market entry was even higher, and payoffs were negative in most rounds. This suggests the self-selected entrants were overconfident in their skill due to what Camerer and Lovallo call “reference group neglect”. The participants seem to neglect that the others in the reference group also self-selected in to the experiment and think they are skilled too.

What type of overconfidence might the market entrants by showing? Could Moore and Healy’s model explain the behaviour in this experiment?

Moore et al. (2007) ran an experiment on firm entry and found, like Camerer and Lovallo, that entrepreneurs overweight personal factors and underweight competitors when making entry decisions. However, when they varied the task difficulty, they found excess entry only when the industry appeared an easy one in which to compete. When it appeared difficult, too few entered.