Central Perk: Seth's Blog : Avoiding false metrics

duminică, 6 mai 2012

Seth's Blog : Avoiding false metrics

Avoiding false metrics

At the local gym, it's not unusual to see hardcore members contorting themselves to fool the stairmaster machine into giving them good numbers. If you use your arms, you can lift yourself off the machine and trick it into thinking you're working yourself really hard.

Of course, you end up with cramped shoulders and a lousy workout, but who cares, the machine said you burned 600 calories...

The same thing happens with authors who put themselves and their readers through the wringer to get a spot on the New York Times Bestseller list (more on this here). Danielle LaPorte built a huge campaign around putting her book on the list, she succeeded in selling a huge number of hardcover copies in a week (far more than most other books) but didn't make the list because of a secret editorial decision that she's not privy to. At the same time, other authors who do a better job of decoding the secrets end up on the list with far fewer sold.

The point isn't that the list is crooked and unfair (though it is). It measures how good you are at getting on the list, not how many copies of the book your readers buy. The reason to avoid the false metric is that it messes with your shoulders, with the way you approach the work, with the real reason you did the project in the first place.

A third example: many car brands now go to obsessive lengths to contact recent car buyers and ask them to rate their buying experience on a scale of one to five. They use these rankings to allocate cars to dealers, ostensibly to reward the good dealerships. Of course, the dealers are in on the game, and instead of doing the intended thing--providing a great experience--all they do is work hard to get people to give them a five when a drone in a call center makes the call. Many of them will clearly state to a customer, "If anything has happened today that would prevent you from giving us a five when they call, please tell us right now..."

The system of false metrics doesn't create a better buying experience, it creates a threatened customer with pressure to give a five.

And my last example: The Arbitron radio rating system used to rely on diaries in which it asked radio listeners to write down which station they had listened to during the day. Several consultants came along with a service that they guaranteed would raise the ratings of any station that hired them. The secret? Repeat the station's call letters twice as often. It turned out that more repetition led to better recall, which led to more people writing down the call letters which led to 'better' ratings.

A useful metric is both accurate (in that it measures what it says it measures) and aligned with your goals. Making your numbers go up (any numbers--your bmi, your blood sugar, your customer service ratings) is pointless is if the numbers aren't related to why you went to work this morning.