It seems obvious to me that when we face existential threats we should make the best possible decisions. I do this with respect to investment decisions, as well. I don’t rely on “guts” or feelings or luck or hope or faith or hunches or trends. All of those ideas are proxies for some sense of incompleteness in our understanding of probabilities and future outcomes.
So how can we cope with those kinds of uncertainties given existential threats? The core methodology is based on ensembles of predictions. We don’t actually want to trust an expert per se, but want instead to trust a basket of expert opinions—an ensemble of predictions. Ideally, those experts who have been more effective in the past should be given greater weight than those who have made poorer predictions. We most certainly should not rely on gut calls by abominable narcissists in what Chauncey Devega at Salon disturbingly characterizes as a “pathological kakistocracy.”
Investment decision-making takes exactly this form, when carried out rationally. Index funds adjust their security holdings in relationship to an index like the S&P 500. Since stock markets have risen since their inceptions with, of course, set backs along the way, an index is a reliable ensemble approach to growth. Ensembles smooth predictions and smooth out brittleness.
Ensemble methods are also core to predictive improvements in machine learning. While a single decision tree trained on data may overweight portions of the data set, an ensemble of trees (which we call a forest, of course) smoothes the decision making by having each tree become only a part of the final vote for a prediction. The training of the individual trees is based on a randomized subset of the data, allowing for specialization of stands of trees, but preserving overall effectiveness of the system. Deep learning methods share similar characteristics.
Weighting and aggregating polls a la FiveThirtyEight.com is perhaps the best example of applying this same methodology in the political realm.
Where do these approaches break down? Inflection points and sudden crises like global pandemics. These so-called “black swan” events are by nature unpredictable but are typically short-lived, so they distort the smoothness but don’t otherwise have significant, long-term impact. In the investment world where some of these ideas originated, this means that investors hedge against volatility with sophisticated mechanisms like the CBOE VIX index that, again, uses an ensemble approach to predicting how unpredictable the markets may become.
Now, for existential crises like global pandemics and global climate change, all we need is an ensemble of experts who are trusted and who smooth out the predictions and prescriptions enough that we don’t devolve into a truly failed state instead of just a weaselly kakistocracy.