luni, 31 octombrie 2016

Seth's Blog : What does the poll say?

It says that people don't understand polls. Even smart marketers get it wrong. What do people think? There's a lot of confusion, much of it intentional, some spawned by a presumed fear of simple math, all of it worth clearing...

What does the poll say?

It says that people don't understand polls. Even smart marketers get it wrong.

What do people think? There's a lot of confusion, much of it intentional, some spawned by a presumed fear of simple math, all of it worth clearing up. 

A survey is not a poll is not a census. A census is what you get if you ask every single person what they think or who they are. There are only two reasons to have a census. Either you want each person to feel personally involved (hence an election) or you are keeping track of each person's answer. For example, if you're printing up t-shirts for the Frisbee team, you ought to do a census of the team to find out what size each person wants, then deliver each person the size they seek.

You could do a survey, which is merely a collection of answers from whomever cares enough to answer the survey. A survey is a useful tool for brainstorming, but it shouldn't be confused with what the group actually feels. Your lack of rigor in setting it up is repaid in a lack of precision in the data.

And a poll? A poll is a smart shortcut, a statistical method for replacing a census (asking everyone) with a very close approximation achieved by asking the minimum number of people required to get a useful answer. A properly done poll will get you an answer nearly as useful as an accurate census will, without the expense or the time.

It rarely makes sense to ask all of your customers about how they feel. You're wasting their time (and yours) by adding more entries into the database without those entries actually making the database any more accurate—part of the problem is that the only people who answer surveys are annoyed or have nothing better to do, and simply making a poll bigger doesn't make it better.

When big companies ask you to fill out a quick survey after talking to a customer service rep, they're not actually doing a survey. What they're doing is snooping on their customer service people, and your answers are directly connected back to each rep, so that person can be scolded (or worse) if they do a bad job.

A poll doesn't predict the future. The media has completely missed this point, again and again. If, on the day the iPhone was announced, you had done a well-designed poll of adults and asked, "Do you intend to ever buy a smartphone?" the yesses would have certainly been less than 5% of the result.

Of course, a decade later, that's turned out to be completely wrong. Was the poll in error?

No.

An accurate poll is a snapshot of right now, based on what's happening today. That's all. If outcomes end up being different a week or a year later, that's not the poll's fault, it's our mistaken belief that the future can be predicted.

To go one step further, the question that gets asked is as important as the answer. Try this at home: When you ask people a question, they rarely give you the straight up truth in their answer, especially when there are social factors at play. The very best polls combine not only the right math, but more important, the right question structure.

The magic of sample size. Let's say you had a bag of M&Ms. You know they come in six colors and you want to figure out the percentage of each that's in the bag. As long as the candies are distributed within the bag, it turns out that no matter how many are in the bag, whether it's a 2 pound bag or a 2,000 pound bag, all you need to do is randomly pull out 300 to 400 M&Ms. That's plenty. More samples won't dramatically increase the quality of this poll.

The purpose of the sample is to pick a random selection from a coherent group.

The key to this is understanding that sample size is relevant for any sized group that's consistent in its makeup. As soon as you can divide the group into buckets, you benefit by doing multiple samples.

Most of the well-done polls you hear about in public do not have a sample size problem. It's a red herring.

The power of bucketing. But what happens if you realize that there are more than one kind of M&Ms, and that different kinds have different color distributions? (This, it turns out, for mysterious reasons, is true. Almond M&Ms only come in five colors).

Well, you could take this into account and run much bigger sample groups, or you could get smart about sample size.

It turns out, for example, that women who ride Harley Davidson motorcycles want different things from them than men do. It also turns out that (I'm guessing about all the Harley stats here) perhaps 10% of the people who buy a Harley are women.

Given that, you could poll 300 women (the easy minimum) and then 2700 men (so you get the balance right). OR, you could get smart, and poll 300 women and 300 men (because every time you add a new person, it's really expensive). "But wait," you might say, "that's not right, because women are overrepresented."

So far, that's true. But after you figure out how women think, and then figure out how men think, you can weight the men's results in your final tally. If, for example, you discovered that women intend to buy a new Harley every two years, but men intend to buy one every six years, you could then report back that the average customer intends to buy a new Harley every five and a half years or so.  (Said with full knowledge that it's dangerous to average averages, but in this case, it's correct.)

Confusion about polls is easy. And the more we try to make decisions using polls, the more careful we need to be about the structure and motivation of the poll itself. 

But finding an accurate poll is pretty easy as well. Most pollsters (in private and in public work) are transparent about their methods, and the magic of statistics is that the math of how the poll is structured can be checked by others. 

Too often, marketers do surveys, not polls, or bother everyone with a census, poorly done. Worse, they then use these results as an accurate prediction of the future, instead of a reliable snapshot of now.

It's the surveys that are so often wrong, deceptive and confusing. It's surveys ("no one I know believes that") that feel like they're accurate but rarely are.

And if we're going to challenge a poll, far smarter to challenge the questions ("that's designed to get the respondent to lie") or the flaws in sampling ("this requires all polled individuals to have a home phone, but of course, an entire generation of young people don't have one.")

But it makes no sense at all to throw out the results of polls we disagree with. The quality of the cars we drive, the efficacy of the medicines we take are all directly related to the very same statistical techniques that we use to run a poll. Ask the right questions to the right people and your snapshot is going to be helpful.

If you want to, be wary of polls. But be wary for the right reasons.

       

More Recent Articles

[You're getting this note because you subscribed to Seth Godin's blog.]

Don't want to get this email anymore? Click the link below to unsubscribe.



Email subscriptions powered by FeedBlitz, LLC, 365 Boston Post Rd, Suite 123, Sudbury, MA 01776, USA.

duminică, 30 octombrie 2016

Seth's Blog : On your toes

In any given meeting, on any given day, most people are merely showing up. It's the 50th time he's performed that sonata. The guy in the outfield had a hard day at home before the game. The folks in the...

On your toes

In any given meeting, on any given day, most people are merely showing up.

It's the 50th time he's performed that sonata. The guy in the outfield had a hard day at home before the game. The folks in the meeting are realizing that it's yet another meeting in a long series of meetings, and wondering if it much matters anyway.

Every once in awhile, though, someone is on their toes. This cocktail party is a big deal, he thinks, because he's going to meet that agent that could change everything. It's the key presentation before launch. This performance in Carnegie Hall is... well... it's Carnegie Hall. 

We can't be on our toes all the time. It's too exhausting, and we can't keep it up.

But what happens if we decide, everyone in this room, right here and right now, at least for a little while, that we'll act as if it's the first time, or the last time, or our best shot?

What would happen if we all got on our toes, together? Just for a little while?

That's when big things happen.

       

More Recent Articles

[You're getting this note because you subscribed to Seth Godin's blog.]

Don't want to get this email anymore? Click the link below to unsubscribe.



Email subscriptions powered by FeedBlitz, LLC, 365 Boston Post Rd, Suite 123, Sudbury, MA 01776, USA.

sâmbătă, 29 octombrie 2016

Seth's Blog : Decoding pro wrestling

Professional wrestling is fake. The blood is fake, the lack of physics is fake, the arguing with the ref is fake, the rivalries are fake... it might be professional, but it's not real. This willful disregard for reality is at...

Decoding pro wrestling

Professional wrestling is fake.

The blood is fake, the lack of physics is fake, the arguing with the ref is fake, the rivalries are fake... it might be professional, but it's not real.

This willful disregard for reality is at the heart of pro wrestling. It's a juvenile fantasy, come to life. An opportunity to make up the rules, ignore authority, and exert bullying force on others, merely because you can.

So why is Billy Corgan (of Smashing Pumpkins) one of the most successful musicians of our generation, running a pro wrestling organization?

He says it's because it's one of the last transgressive arenas left. That it's a morality play, a microcosm of the human condition, a chance to put on a show that highlights our fears and our avarice. He knows that it's fake, authenticity is a foreign concept in this world.

And what lesson can we learn from politics importing pro wrestling's mindset? Once you see it, you can't unsee the connection. Worth noting that one of the key narratives of pro wrestling is the fake within the fake--someone is always claiming that the outcome is rigged. (In wrestling, of course, it is rigged. And so is the complaining.)

Pro wrestling works as a play and a medium because the people who are part of it realize that it's fake.

It turns out that modern media is a perfect match for the pro-wrestling approach. You can put on a show, with your own media, as often as you like. And that show is, to many, remarkable, and so it spreads.

And there lies the danger, the opportunity for pro-wrestling thinking to corrupt our society: When the fans, or worse, the participants, don't realize that it's fake.

In real life, the laws of physics actually work. In real life, blood feuds rarely end well. In real life, accepting the ref's decisions is the only way to have civilization...

The filter bubble creates an echo chamber, and reality stars are pushed to be more like cartoonish pro wrestlers and less like responsible human beings. If it bleeds, it leads.

You probably work with people who are living in their own pro wrestling universe. These are people who are so in love with their version of reality and their goals that they view the real world as an affront, an intrusion on the way they insist things turn out.

Reality remains our common ground, the best one we have to work with.

       

More Recent Articles

[You're getting this note because you subscribed to Seth Godin's blog.]

Don't want to get this email anymore? Click the link below to unsubscribe.



Email subscriptions powered by FeedBlitz, LLC, 365 Boston Post Rd, Suite 123, Sudbury, MA 01776, USA.