20  Model complexity

There are markedly more complex analytical methods than reference class forecasting, involving vastly greater data, with which to make decisions. However, there is actually much power in simple analytical techniques.

20.1 Improper linear models

In a 1979 paper Robyn Dawes (1979) demonstrated the power of “improper linear models.”

A common statistical procedure when developing a model is a process called multiple regression. Each of the variables and outputs are entered into a formula that determines the optimal weighting of each input variable in developing the output.

Dawes showed that a model that combines each of the variables with equal weight, rather than optimal weight, does just as well as multiple regression in many instances (and, as per the earlier examples, better than the humans they are matched against).

A large driver of this is the bias-variance trade-off we discussed earlier in the unit. The improper linear model with equal weights is biased, but it is less affected by random variation in the data that is used to develop it. It is biased but has lower variance than multiple regression.

20.2 Simple heuristics

In Simple Heuristics That Make Us Smart, Czerlinski et al. (1999) describe a competition between some simple heuristics and multiple regression. Both were to predict outcomes across 20 environments, such as school dropout rates and fish fertility.

One simple heuristic in their competitions was “Take the Best”. This heuristic operates by working through variables in order of validity in predicting the outcome. For example, if you want to know which of two schools has the highest dropout rate, you ask which of the many possible predictive cues has the highest validity. If attendance rate has the highest validity, and one school has lower attendance than the other, infer that that school has the higher dropout rate. If the attendance rate is the same, look at the next cue.

Depending on the precise specifications, the result of the competition across the 20 environments was either a victory for Take the Best or at least equal performance with multiple regression. This is impressive for something that is less computationally expensive and ignores much of the data (in other words, is biased).

Czerlinski, J., Gigerenzer, G., and Goldstein, D. G. (1999). How good are simple heuristics. In G. Gigerenzer, P. Todd, and The ABC Research Group (Eds.), Simple heuristics that make us smart. Oxford University Press.

Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571–582. https://doi.org/10.1037/0003-066X.34.7.571