In a recent interview former Major League Baseball all-star pitcher Dwight Gooden tells this story about when he was trying to make a late-career comeback:
“You have to remember all the new stadiums they got the radar guns and they’re turned up three or four miles an hour faster just for the fans.
And unfortunately I found out the hard way. I went to Cleveland towards the end of my career and when I left the Yankees in ‘97 I probably was topping out maybe 92, 93 (miles per hour). So I go to Cleveland and pitching at home and the game – the clock’s got me at 96, 97. I’m thinking, ‘Wow. My fastball’s back.’ Unfortunately, three minutes later I’ve given seven runs, four homeruns, who knows how many hits. So now when I get knocked out of the game I go in the room with the guy that keeps the video and I’m looking at my chart and seeing who did what and then I see like 87, 88, 89 and I’m asking, ‘What are these numbers here?’ He says, ‘That’s your velocity.’ I say, ‘Out there it had me throwing 97.’ He said, ‘No. That was turned up for the fans.’ So I found out the hard way.” [condensed by removing minor asides from the interviewer]
This is just one of many, many cases in which data is being deliberately falsified to serve a particular purpose.
In an excerpt from his book “The Signal and the Noise”, Nate Silver talks about weather forecasters deliberately overstating the likelihood of precipitation:
For many years, the Weather Channel avoided forecasting an exact 50 percent chance of rain, which might seem wishy-washy to consumers. Instead, it rounded up to 60 or down to 40. In what may be the worst-kept secret in the business, numerous commercial weather forecasts are also biased toward forecasting more precipitation than will actually occur. (In the business, this is known as the wet bias.) For years, when the Weather Channel said there was a 20 percent chance of rain, it actually rained only about 5 percent of the time. People don’t mind when a forecaster predicts rain and it turns out to be a nice day. But if it rains when it isn’t supposed to, they curse the weatherman for ruining their picnic.
And one more example: I have a Hyundai Elantra. When the fuel gauge says that the gas tank is half full, in fact about two-thirds of a tank is left. And when the fuel gauge shows that the tank is almost empty and the warning light comes on, there’s still about a third of a tank left: enough to go over 100 miles in the city, and over 200 miles on the highway. I need not panic.
In each of these cases, the holder of the data had what they felt was a good reason to deliberately mislead the user of the data. But the consumer of the data may have a different agenda. (If you’re a farmer, or in a drought-stricken area, it may be more disappointing that it doesn’t rain when you expect and want it to.)
These may seem like minor examples, but this deliberate falsification may come into play in far more important situations, too. For example, if a marketing agency tells you that on average with their landing page optimization service they have increased the leads and sales for companies by 43%, can you trust them? Does it vary by industry? What’s the median?
Am I suggesting that some companies would put out false numbers about the performance of their products and services? They’ve done worse. When a friend of mine was attending Harvard Business School I asked him how it was going. “Great,” he responded. “I’ve already learned nine ways companies can hide profits.” Or can appear more profitable.
In the scientific world, new discoveries are vetted in peer review journals. (Sometimes…) The scientists must allow full access to their methods and data for others to review. However, even this system is being strained as the amounts of data that experiments are producing are becoming so enormous. The Large Hadron Collider produces over 25 petabytes of data each year. Scientists are re-evaluating how to store data, and make it available, and who’s going to pay for it all.
The world of marketing and sales is not nearly so rigorous, or open, as the scientific world. You rarely have access to the original data. So when it comes to third-party data, keep your skepticism on.
Did you think this was interesting? Then sign up to get our daily email marketing tip that people call “very impressive and helpful”
