All models are wrong, but some are useful.
— George E.P. Box, past president of the American Statistical Association
As I write this, a search of my Twitter timeline for “poll” turns up seven tweets about six different polls, including a teaser tweet from CNN for a poll about to be released — in just the past hour.
The media’s and the political blogs’ reporting of polls — and speculation about what they mean — has been more pervasive and inescapable this year than in an election I’ve ever seen, even though no one knows what these polls mean for the election.
On Friday afternoon, the “Latest Election Polls” page on the website RealClear Politics had four new national general election polls posted. Here are the results of those polls:
- Obama +3
- Obama +2
- Romney +6
Confused? The CBS News website assumes you are, and published a piece Friday headlined “Confused by all the polls? Pollsters explain the variation.” It doesn’t help.
Not only are we being showered with words and more words (and charts that may or may not be meaningful) about each and every poll that’s released, we’re presented with a daily parade of stories, blog post and tweets about the averages of a bunch of polls and statistical models (whose methodology is not made clear) built from polls and other factors: RealClear Politics Poll Averages, the CNN Poll of Polls, HuffPost Pollster, the Talking Points Memo Poll Tracker, and the most influential of them all, the New York Times' FiveThirtyEight blog.
This can lead to some silly journalism. The website Business Insider ran a story on Friday about the RealClear Politics Poll Averages showing Obama up by 0.1 percent with the headline “WAIT! Obama Has Re-Taken The Lead In An Average Of National Polls.”
Christopher John Farley, who writes the Speakeasy blog for the Wall Street Journal, has had enough, and at least he has a sense of humor about it.
“Speakeasy’s own internal polling suggests that there are way too many presidential polls. Our sample size is one – me – so our follow-up poll – also of the same person – advises that you probably shouldn’t trust the results of that first poll too much,” Farley wrote a couple of weeks ago. “Both surveys have a margin of error of plus or minus 100 percent.”
Who is to blame?
Twitter: Looking for a place to tell the world your poll is coming out, that your poll has come out, what your poll says? Looking for a place for one-sentence analysis of that poll that just came out, proving your candidate is going to be the next president? Looking for a place to disparage in one sentence the method of that poll that just came out? Get yourself to Twitter.
Nate Silver: The author of the New York Times’ FiveThirtyEight blog has become the Wise Man of political prognostication — and he posts to his blog pretty much every day. Traditional media political reporters, bloggers, tweeters and your uncle all love to quote Nate Silver. Silver himself, interestingly enough, is as capable of misreading the politics of the moment as any blogger (or your uncle), as this Sept. 28 interview with New York Magazine demonstrates.
Political reporters: Covering politics can be complex. Candidates make intricate policy proposals, debates between candidates can be full of obscure and arguable numbers and, worst of all, the candidates go through their days on the campaign trail giving essentially the same speech over and over and over again. Reporting on a poll is like reporting on a baseball game. The news is simple: someone’s winning, someone’s losing, and the numbers have changed since the last poll. A smart reporter or pundit, of course, will remind everyone that polls are “just a snapshot.” The Daily Show’s Jon Stewart summed what we all want to know about the incessant coverage of polls: “The problem with the polls are that they are terrible, unreliable and invaluable. How the hell are these polls so divergent, useless, yet accurate and quotable?”
Partisan blogs: The political blogs are the headquarters for Making A Poll Mean Anything You Want It To Mean. Polls are just jumping off points for political bloggers to shout into their partisan echo chambers, whether on the right or the left.
College statistics courses: Anyone and everyone who has successfully completed a Stats 101 course (including, just barely, me) is certain he or she can read the poll’s method section and instantly find its fatal flaw — and then tweet it or post it to a blog or a discussion board. Ha! Told you! Recall the 1711 couplet from the poet Alexander Pope: “A little Learning is a dang’rous Thing/Drink deep, or taste not the Pierian Spring.” The emphasis there is on “little.”
I am probably last person you want to turn to for knowledge about statistical sampling, but, if you're interested, the website electoral-vote.com has a useful FAQ about how political polling is done, and the website stats.org clearly explains “margin of error.” Carl Bialik, who writes The Numbers Guy at the Wall Street Journal, wrote an article during the last presidential election about the dangers of poll averaging called “Election Handicappers Are Using Risky Tool: Mixed Poll Averages.”
Hold on a minute — Monday morning brings the results of a new poll. The lead story in this morning's Wall Street Journal is "Dead Heat for Romney, Obama."
What a surprise. If you have a moment, go back and click on that link two paragraphs above to the explanation of margin or error in political polling. This race has been a dead heat for two weeks. No game-changing news there.
But if you have a keyboard, an internet connection and an opinion, the only fun in writing about a tie game is speculating who might win. Be prepared for two more weeks of poll parsing and specious speculation.
Follow @mleccese on Twitter.
The author is solely responsible for the content.