American Culture

Pew Research sees different polling results when cell phones are included

My wife’s younger siblings don’t have land lines. Several of my friends don’t have land lines either. Their cell phones are their only phones, and from what I’ve read, this is entirely normal for people my age and younger. So I’ve read claims that land line-only (LLO) polling is as accurate as land line plus cell phone polling with a bit if disbelief. And so it’s with great interest that I read a blog that hit my screen this morning from Future that the Pew Research Center has identified a 2-3% shift in results in three Pew polls that were conducted twice each, one land lines only and the other land lines plus cell phones.

Polling organizations have been worried about the difference in results between the two different polling techniques for two basic reasons. The first is accuracy. As the Pew Research paper says, “U.S. government surveys estimated that about 15% of adults were ‘cell only’ in the fall of 2007 and the rate of increase since 2004 has been at least 2% a year, meaning that the number may be as high as 17% in this election cycle.” That’s a lot of adults who are being ignored in the polling data. Weighting factors to correct for statistically known differences between groups only work if the group being adjusted for has the same views on the poll as the groups actually polled. However, when the cell phone only group tends toward support for Obama over McCain, then the weighting factors used are simply wrong.

The second reason is cost. Calling land lines is cheap because polling organizations use computerized dialing programs. Using computerized dialing to dial cell phones is illegal, so polling organizations have to pay real people minimum wage or better to call each cell phone manually.

Michael Connery at is understandably concerned:

[I]f young voters are being underrepresented due to their cell-phone habits and likely voter screens, what will that mean for the accuracy of the polls leading up to election day? Could the pollsters be as wrong in 2008 as they were in 2004?

The unfortunate answer is yes, the pollsters could easily be that wrong. In fact, given that there are about 8% more cell phone-only adults in 2008 than there were in 2004, the pollsters could be significantly more wrong than they were in 2004.

It’s good to know that many polling organizations are including cell phones when they perform their phone surveys. But I haven’t yet found a list anywhere of which polling organizations include cell phones and which ones don’t. Knowing this information alone might make the difference between whether a poll should be taken seriously or not.

UPDATE: As per Gericault’s comment below, has a list of the polls that are compensating for the cell phone effect, or have started doing so recently, and they are as follows: Pew, Gallup, USA Today/Gallup, CBS/NYT, Time/SRBI, NBC/WSJ, ABC/Washington Post, AP/GfK, the Field Poll in California, PPIC, and Ann Selzer. The following polls do not compensate for this effect, but may be “OK” due to other methodological factors: Internet-based pollsters, Zogby Interactive, Economist/YouGov, and Harris Interactive.

Ultimately, though, Pew’s response to this new data is the only correct response:

[T]he Pew Research Center for the People & the Press will include cell phone samples in all of its remaining election polls.

11 replies »

  1. Yup – a 2-3% error in McCain’s favor. So we have to factor in that and the closet racist factor, as well, and that may be 5% in the other direction.

    Fun, fun.

    My wife and I don’t have a land line, by the way.

  2. If these younger voters who are cell phone dependent can be counted on to turn out for the election, Brian, the figures I’ve seen suggest an advantage to Obama of several percentage points.

    Let’s hope those figures are right….

  3. Thanks for this, Brian. It’s a small difference, to be sure, and there are some massive questions about how many of this youth vote will actually turn out. But it does raise some very, very interesting questions about demographics in the swing states, doesn’t it?

    Young voters, I’m guessing, tend to be urban (and better coverage for cell phones in urban areas would tend to make cell-only use more practical than in rural areas). Urban areas already vote for Democrats, on average, so the real impact, if there is any, may be in those states with sharp voting splits between urban and rural voters: Colorado, Oregon, Washington, and Virginia come to mind. If there really is a cell-phone effect, it might suggest that Obama will win Oregon and Washington, and may just win Colorado and Virginia. Virginia would be a stunner to me, and may well put him over the top.

    Thanks again.

  4. Sam – My wife and I have considered going without a land line, but we’re presently getting a great deal on DSL, phone, and DirecTV bundled together, so purging the land line would actually cost us more money right now, not less. And while she was a teacher, a land line for irate parents to call made the most sense – no-one wants irate parents and obnoxious kids to have access to a teacher’s cell phone number.

  5. Brian,

    My wife and I are also getting a good deal having cable, broadband, & phone packaged but we have the land line primarily as a back-up. the ringers are off and neither one of us knows the number by memory. We use our cells phone almost exclusively.