Superforecasting: The Art and Science of Prediction
There is a huge market for forecasting in our country, from political talking heads on TV, to investment advisors, to the approximately 20,000 intelligence analysts working in our intelligence establishment. But, while the stakes for getting it wrong can be huge (see Iraq’s WMD), there is no formal reliable way of measuring or analyzing the track records of those doing the predicting. Pick your favorite commentator or columnist, what’s their hit rate on their predictions? That’s impossible to answer, first because no one has compared what they said would happen to what did happen, and even if they did, so many of their predictions are so vaguely worded that most of them can easily claim they meant something else and wiggle off the hook.
Philip Tetlock is trying to change that. Beginning in the 1980s, he has been studying how good experts are at prediction (answer, just slightly better than a drunk monkey throwing darts). One of his findings was that pundits who were the most confident tended to be wrong more often, but they also got on TV more often. They are hired more for their ability to tell a compelling story with confidence than for their track record in forecasting.
This latest book details his findings from a four-year project funded by IARPA, the Intelligence Advanced Research Projects Activity, to test the forecasting performance of several different teams of experts. It was a large test which asked over 500 questions to more than 20,000 participants between 2011 and 2015. It was also rigorous, with questions designed to eliminate the wiggle room problem. For example, they asked, “Will any country withdraw from the Eurozone in the next three months? How many additional countries will report cases of the Ebola virus in the next eight months?”
The study found that about 2% of participants, which he calls superforecasters, are consistently more accurate in their predictions. By identifying the superforecasters, and then testing different combinations and variables, he was able to tease out what makes them successful, and the bulk of the book explains the traits, techniques and habits of thought that make for superior judgment.[1]
The basic theme is that it’s not superior intellect that distinguishes the SFs, but how they think. Here are just a few of his recommendations:
- Break down tough problems into their components, and make estimates or judgments about those.
- Pay attention to base rates first, and then adjust. For example, I may think that my friend is very likely to strike it rich in a very difficult venture, because I start with knowing how smart he is. But if I begin by considering that the odds are 50 to 1 against success, I could double his chances and still think it’s very unlikely.
- Be actively open-minded, not only being open to new information but looking for it. Once you have formed a judgment, pay attention to new information, especially anything that would call your initial judgment into question.
- Write down your forecasts and your reasoning, because the mere fact of writing it will help distance you emotionally from your first prediction. If it’s important that you get it right, take the further step of writing down all the reasons you might be wrong, and then synthesize the two.
- Expand your range of alternatives. Most people have a three-position dial about predictions: yes, no, and even odds. You can force yourself to become more systematic about your own thinking by adopting a 7-point scale as recommended by the National Intelligence Council as you see here:
Remote Very unlikely Unlikely Even Probably, likely Very likely Almost Certain
Even better, use percentages. It won’t guarantee[2] you’re right, but it will force you to examine your own thinking and help you adopt a more nuanced viewpoint.
There’s far more good advice than I can summarize, but frankly I’m struggling a little in deciding whether to recommend that you read Superforecasting. On the plus side, I predict that it is very likely that if you read and apply its lessons, you will become a better thinker. On the other hand, it’s an even chance that you will become a worse persuasive communicator. That’s because an effective salesperson radiates confidence about the future they recommend, while effective forecasters are far more cautious and humble about their predictions.
My personal choice would be to begin with better thinking. First, for the obvious point that you owe it to yourself and to others to give them your best thinking. Second, sustainable influence depends on credibility, which will in the long run correlate strongly with your predictions. It’s true that TV pundits who are the most confident in their predictions tend to be wrong most often, and they don’t suffer for it. But when people are putting their own reputations or money at stake based on what you predict, they tend to have longer memories.
[1] I use judgment in the sense that they are better predictors, not that they necessarily make better decisions.
[2] In fact, you may have noticed that the seven-point scale does not include certainty on either side.
I’ll get to my review of Mike Weinberg’s book, Sales Management. Simplified.: The Straight Truth About Getting Exceptional Results from Your Sales Team
Despite spending big in free agency during the offseason to acquire players like Ndamukong Suh, and spending lavishly on high tech tools such as GPS trackers and sensors to analyze practices, the Miami Dolphins have won only one game this season, so they fired their head coach earlier this week.
Enter a new interim head coach, Dan Campbell, who was promoted from his position as the tight ends coach and who looks like he could still suit up and play today. They held their first practice under the new coach yesterday, and it was definitely a different approach. Almost the entire practice consisted of one-on-one drills, where players went at each other in full pads. Campbell’s aim is to make each of his players “violently compete” during practice, and his description of practice afterwards was priceless:
“Everything else was all about being primates again. Every one of those guys. Just back to the days where, hey, you line up and you go.”
What does that have to do with Sales Management. Simplified.? Think of the Dolphins as a sales organization that needs to change its ways if it wants to get back to winning ways—Campbell and Weinberg appear to be “twin sons from different mothers”, as it relates to how to get the best out of people who work in a performance-oriented contact sport.
I’ll give you just two examples to illustrate the wisdom and practical advice contained in the book. The first is that sales managers have to get out from behind their desks and go right to the action to observe and coach their players.
The Dolphins have been operating like many sales organizations today, by over-relying on fancy measurement tools. They hired a director of sports performance and director of analytics, and equipped each player’s shoulder pads with GPS trackers and sensors that measured energy expenditure, distance traveled and speed during practice, temperature, etc. At the end of every practice, the numbers are analyzed, and presumably the coaches would use the data to make adjustments and refinements in a constant quest for improvement.
All that data, analysis and tweaking has not prevented a 1-3 record, a statistic which hides how embarrassing their play has been. They haven’t just lost three games out of four: it’s the way they have lost. They have been quite simply outplayed and outmuscled on a man-to-man basis at almost every position.
The key lesson for sales managers is that all the fancy strategy, tools, and measurements are great, but completely useless unless you get the basics right. Sales is just as much a contact sport as football, where success is a matter of what happens when a salesperson sits across the desk from a prospect. In sales, just as in football, “you line up and you go”. This may sound crude, but it’s still two primates facing each other, speaking, listening, and presumably influencing each other.
Too much focus on shiny new toys such as fancy sensors and measuring tools can take away from the best tracking technology a sales manager has: their eyes and ears. As Weinberg tells us, those basic tracking tools are best deployed in three ways: coaching salespeople one-on-one, conducting productive sales meetings, and riding with salespeople in the field to directly observe what’s going on. How often are you getting out into the field and actually observing what goes on in sales calls? What are your salespeople actually doing and saying when in front of customers? How are customers reacting?
The second lesson is that culture counts—a lot. Weinberg spends the first half of the book describing dysfunctional sales cultures, where there is little or no accountability, sales is seen as a poor stepchild to other corporate functions, and various other symptoms that you might recognize from sales cultures that you may be in.
The best sales cultures are competitive, blunt, accountable, focused on selling, and relentlessly self-improving. These “shared attitudes, values, goals and practices” are what turns a collection of individuals of diverse talents into an effective team. Coaches can’t be everywhere at once, and culture provides the guidance that drives performance.
There’s a lot more in Mike’s book than I can write about here, so whether you’re a beginning sales manager, an experienced sales manager trying to improve an underperforming team, or a successful sales manager who wants to sustain your success, this book is for you.
Now, if he could just do something for my poor pathetic Dolphins…
The folks at CEB have done it again—written a book that challenges traditional thinking about B2B sales and introduced a new character in the long-running conversation about understanding and influencing the customer’s decision making process.
Trying to describe the ideas in The Challenger Customer: Selling to the Hidden Influencer Who Can Multiply Your Results
In that spirit, The Challenger Customer is about helping your customers buy. In sales, we lament how hard selling is nowadays; buyers have far more knowledge earlier in the sales cycle and use it to drive even complex solutions to commodity status. The problem with that is that often it’s not in the buyer’s own best interests to buy the lowest-cost solution, yet many buyers make the sub-optimal decision because they can’t help it: buying is harder than ever before.
Buying is harder because more stakeholders are involved: an average of 5.4 stakeholders in complex B2B deals, according to the book. That’s complicated by the fact that the most important attribute that senior decision makers consider when choosing a supplier is widespread support across the organization.
The traditional sales response to this challenge is to simply work harder. If you need to get more yesses to close the sale, you just have to call on more people and get their buy-in, right? The revelation—at least to me—is that, that strategy will actually make it less likely that you will get the sale. In other words, 1+1+1=0! That’s because each stakeholder will support the deal for their own reasons, and the overlap among interests becomes harder to achieve as the number of stakeholders rises. As a result, the decision gets driven down to the lowest common denominator: either status quo or the simplest, cheapest choice.
The challenge, then, is not to get a serial collection of yesses, but a collective yes, in which each stakeholder converges around a common vision. It’s like the parable of the six blind men and the elephant. Each one sees only a small part of the whole, so someone needs to make them see the whole elephant. That’s a daunting task for any salesperson, but fortunately there’s a solution: enter the Mobilizer.
The Mobilizer is the internal Challenger, the person who is willing to make waves to and drive the vision. They will only do it if they perceive that the pain of staying the same is greater than the pain of change. The book explains in great detail how to identify the three types of mobilizers, get them to agree on the need for change, and then coach and equip them to sell the need internally.
I give The Challenger Customer five stars for three reasons:
- It’s very much about outside-in thinking. Start from the customer’s perspective, understand their need to change, and don’t lead with your product.
- Just like their first book, The Challenger Sale, it’s backed up by tons of primary research, very credible examples, and detailed implementation suggestions.
- The third reason is why I didn’t like the book on first reading, and then I did: the approach and techniques are devilishly difficult. You have to learn how to identify mobilizers, tailor your approach to each of the three types, help them get the message across effectively to the other stakeholders, produce the right materials, and a host of other challenges. But by the second reading, I realized that the difficulty is actually the best reason for a company or even an individual sales rep to adopt the approach. If it were easy, anyone could do it, and then it would not be an advantage anymore.
That said, this is not really a book for salespeople. Only a select few (like the type who reads this blog, wink-wink) would be able to master the techniques. It takes a joint effort by sales and marketing to generate the insights and produce the materials to equip the Mobilizer to sell the insights internally, and it won’t happen overnight.
I suggest you read this book, study it, challenge it, and most importantly, use it to change the way you sell.
This blog post could change your life—maybe even prolong it.
If you get stage fright before a presentation (and who doesn’t?), if you are under stress (and who isn’t?), if you think stress is bad for you (and who doesn’t?), then I strongly recommend that you read and take to heart the central message of The Upside of Stress: Why Stress Is Good for You, and How to Get Good at It
Whether you think stress is bad for you or good for you, you’re right.
To put that last statement in a less cryptic way: stress can improve your performance, make you stronger, and even make you a more caring person—as long as you believe it can. In fact, the best way to succeed in stressful situations is not to try to reduce your stress, but to embrace it as a resource to propel enhanced performance.
I realize that sounds like superficial motivational hooey, brought to you by the same people who tell you that you can do anything you set your mind to, but McGonigal, a psychologist at Stanford, backs up her assertions with extensive research and a few eye-opening studies.
In some ways, The Upside of Stress does not tell us anything new. We’ve all heard the meme that what does not kill you makes you stronger, and I have long been telling students in my presentations classes that anxiety before a speech means that you care and that you are gearing up for superior performance. So, yes, we have heard some of this before, but this is the first book I’ve come across that backs that up with research and explains the biology behind these ideas.
What is also new is that we learn that there is more than one possible response to stress. We’ve been taught that stress is caused by the activation of the fight or flight response in our minds and bodies. That response is a natural reaction to threat, which prepares our minds and bodies for superior performance, but it evolved many millennia ago in a far different environment than our modern world. So, according to the mismatch theory, our stone-age brains respond to modern circumstances in ways that can hamper performance and over time can severely damage our health.
That makes sense if fight or flight is our only option, but McGonigal explains that there are actually three different possible responses to stressful situations. Besides the familiar threat response, we can have a challenge response or a tend-and-befriend response. Although both possible responses are equally important to well-being, my focus in this blog is on the challenge response.
The difference between the threat response and the challenge response lies in our estimation of our ability to meet the situation that faces us. When we’re fearing for our life, our body does the sensible thing: it goes into defensive mode and sends out hormones that cause a lot of changes; one of the most important is that it constricts blood vessels around our heart, because it might reduce blood loss in the event of severe injury. When we’re not in fear, different hormones cause the blood vessels to relax, which allows for greater blood flow and more energy to rise to the challenge and drives better performance, not to mention being better for us in the long run.
Evoking the challenge response does not reduce stress, but it does make the stress work in our favor. In studies, it has been shown that simply informing people that stress can help them perform better, can lead to improved performance on standardized tests, for example. One reason may be that the threat response narrows our attention and places greater focus on signs that things are going badly, but the challenge response opens our attention to more positive possibilities and opportunities. In numerous studies, those primed to generating a challenge response through prior education led to better performance. Even better, the benefits tend to last far beyond the initial priming.
So, how do you generate the challenge response? The most obvious first step is to avoid the threat response by creating the conditions so that you are not actually in danger. If you are well prepared for a presentation, you should take comfort in the fact that you are equipped to handle any difficult questions that might come up. (Or as I tell my students, if you’re nervous because you haven’t prepared well, you deserve to be!)
You can also activate the challenge response by viewing the stressful situation as an opportunity for learning and growth. As I’ve written before, this mastery mindset has been shown to improve performance in several different areas, including sales.
Actually, you’ve already completed one of the most important things you can do to generate the challenge response and benefit from stress: simply by reading this article, you are more likely to bring a different mindset to your next stressful situation!
