The Referee's Parable

The Referee's Parable

By Douglas A. Samuelson


"Well, well!" the appliance repairman sang out cheerfully as our hero approached to claim his toaster oven. "Here's the member of the expert tribe who can predict election results. Now it's three days after the election. So who's going to win?"

"Beats me," the analyst admitted. "What a difference a week makes! Last week, when I was telling you about my colleagues who do forecasting models and tracking polls and election night projections, I was pretty confident that their analyses would be right. I can tell you, most of the ones I know aren't happy now."

"Neither is just about anyone else," the repairman laughed. "I heard some Congressmen want to hold hearings on how the networks got it so wrong."

"A couple of my friends, the forecasters, want to investigate, too," the analyst said. "They say they got inconsistent data feeds, maybe because the people doing them were in a hurry and didn't check enough. And the network news people wanted to be first with the news when a state's result was clear, so they kept pressing for a projection when the analysts wanted to re-check the data. Then, once one network made a call, the others' news execs were eager not to look as if they'd missed something  so they followed even though nobody there was sure."

"Hmmm," the repairman mused. "This sounds like the training I got as a referee in my kids' football league. You can confer and pick up a penalty flag if you aren't sure about the call, but if you don't throw the flag, play just goes on. Picking up a flag is embarrassing, but at least that way you get a little more time to think it over."

"I thought people got on your case for calls or no-calls, depending on which side would benefit," the analyst noted.

"They do," the repairman/referee agreed, "but one is harder to correct than the other."

The analyst added thoughtfully, "This reminds me of a problem I had at work a few weeks ago. My boss wanted a recommendation on how to do a study. She had some pretty strong ideas about what techniques would work. I wasn't so sure; some of these 'data mining' tools, for example, can find patterns you'd miss with other methods. The trouble is, although you gain in how easily you can get to some result, you have a harder time interpreting the output later because all those data transformations are tough to explain in plain English. It takes courage, and some credibility with the boss, to be able to say, 'I don't know how to interpret the output of that method to clients' a good no-call, as you'd put it  and look prudent rather than stupid."

"It seems a lot easier to go along with the suggestion and straighten it out later," the referee affirmed. "But to be effective, you have to know when to say you don't know. I took one of my kids to a baseball game some years ago where there was a close play at home plate. The umpire didn't make a move or a sound, and for a few seconds everyone just stared at him, as if to say, 'What's your problem?' Then it dawned on them: He didn't make a call because there was no call to make yet! The catcher had missed the tag and the runner had missed the plate!"

"I wish some managers I know were better about admitting when they don't know," the analyst complained. "I hadn't looked at it this way before, but now I'm thinking that what I've usually seen as arrogance may really have been fear. They felt they had to give some orders, even stupid ones, because that seemed less threatening than having everyone realize they didn't know what to do."

"Makes sense to me," the referee acknowledged. "The coaches who know what they're doing dispute fewer calls, and they don't get as hot when they do argue. The ones who scream about everything may think they're helping their team, but usually what they're doing is convincing us not to listen to them at all."

Another customer had entered the shop and caught the last bit of conversation. "I'm a cop," he interjected, "and I've seen plenty of serious fights. Generally, the one who's doing the screaming isn't the one who's winning."

"We're talking about knowing when you don't know," the analyst explained. "You guys are another good example, aren't you? You want to make an arrest, but you know you'd better wait until you have good enough evidence to make it stick."

"That's what the detectives tell me," the policeman concurred. "I'm not a detective, though. I usually get sent out when somebody reports trouble, typically a neighborhood fight or a loud family disagreement, and my job is to get them to knock if off and quiet down. In most of those cases, if I arrest somebody, everyone else suddenly stops squabbling and they all agree: He's a victim, and I'm the bad guy."

"Your job is like mine," the repairman chuckled. "My refereeing job, that is, not appliance repair. The best I can do is to have everyone forget I'm there. The more they notice me, the worse I'm doing."

"I guess my job is like that, too," the analyst smiled ruefully. "They're most interested in what I do when it turns out wrong!"

Topic #2591 | Report this topic