Superforecasting
Forecasters are just Chips throwing darts at a dartboard, little better than guessing.
Butterfly effect, LaPlasses Deamon, lorenzes cloud. We live in a world of predictability and unpredictability. Prediction accuracy for Forecasts That have a component of chaotic for butterfly affect, will converge towards 0.
forecasts are used for everything from Banks promising financial gain, to pundits saying they know the outcome.
System one and system two in the cognitive reflection test.
We impose order on the world, why we can make System 1 decisions sounds d sane even when there is better evidence. Split brain test, Different hemispheres of the brain are shown different pictures, but the brain creates a story about both of them. sometimes the best positive test is the test where you prove the negative and it turns out to be false.
Any forecasting should clarify different nouns and verbs in the statement. Like The word possible which ranges from 0 to 100%.
Judging judgment. Must have many forecasts to look at judgment. Resolution and calibration. Resolution is how far away from 50/50, calibration is percent of time correct. Briar score.
Big idea forcasters fit everything into that model, doggedly try to fit relativity into their view, worse than dart throwing chimps. Emerald City and green glasses. May increase confidence but not accuracy.
Collective judgment is generally more accurate. Useful information is usually spread out. Min / max cancel each other out. Individual Forecasts may be better. Good forcasters do internal aggregation. Think about other people, rational Vulcan, pseudo radical, aggregators, gut guessers. IE the guess the numbers game.
Reasonable vs correct , don't look at outcomes, look at the way it was analyzed. Can be reasonable and wrong.
Test everything, doubt everything, red teams, AB tests, etc.
Wisdom of crowd is just aggregation of information, once you have this, can increase confidence once you have this.
Randomness can for you, one person Getting 100 coin tosses 70% correct is extremely unlikely, but 2000 people guessing will be close. Similar to the fallacy of guessing the stock market.
Regression to the mean is a powerful concept. in completely random coin toss, if very lucky people got 90% ones they would have more likely hit 10% next time.
Fermi-estimates, Breaking down the question into it’s constituents parts. identifying the knowable from the unknowable. Fermi-zing, and confidence intervals.
Inside view vs. outside view. inside view is data you look at within the event. Outside view is trend or historical data for data related to that event. Better to start with outside to ensure eyou anchor with out being involved in the.
- Fermi-ize
- Outside view
- Inside view based on fermi-ize questions
- Synthesize both
Be Actively open minded, seek other opinions, weight them equally.
Avoid the in, Wide variance and probability can lead people to go into a ignorance Pryor. there is a primal instincts to align with certainty, even though it is against our best System two judgment.
Ailiatory uncertainty (like prericiti g the weather) will always exist.
Porbabilistic thinkers focus on the how it happened vs the why. Be wary of substituting my sorry of beliefs for the one I'm analyzing. IE if person x is likely to make a mistake.
Under or over updating: Achilles heel of forcasters.
Don't need full Bayes equations. Just update priors based on the weight of new evidence.
Groupthink: we cannot all be wrong, don’t mistake agreement for correctness. Ask precision questions, don't question the person or ask vague "what do you mean" unless the person is being ambiguous. Criticism is helpful, I want to hear feedback. Don't Dance and the issues
How to invite criticism more?
Givers, matchers, takers, flow of information and questions are critical.
Contrarian thinking can help inject new information into the group.
"Nothing like Danger to focus the mind" -soros
Distinguish what you are confident about and what you're not. Look part the end of your nose. Self doubt is not the answer, intellectual humility is key, careful reflection, confidence can inspire determinate action.
Don't mistake facts from values, especially when looking at actions by others or organizations.
Kahneman:
scope insensitivity, you imagine the one for the many when valuing something. Need a multiplier. Scope can include time as well!
WYSIWTI - What you see is what there is
Black swan investing == VC
Forecast investing 60% more modest gains.
Perhaps possible to forcasters could sequences of black swan events.