Polling’ firm may have made up numbers; Wis. media dutifully report them
The decline in (or complete lack of) standards for news media coverage of political polling in recent years has been hard to watch.
There was a time when newspapers like the Milwaukee Journal Sentinel would not report on a poll unless it had the entire poll, rather than partial or selective results that are strategically released or leaked to try to shape news coverage of a campaign. The Milwaukee paper would do its own polling.
But those days are long gone. In recent years, as we’ve repeatedly complained, media outlets will report on any poll, from any source, and treat them all as though they are equally valid.
One firm that has raised a lot of questions from political practitioners — but, unfortunately, not from the media — is Strategic Vision, a Republican strategy firm based in Atlanta (and, it claimed, in Madison). Strategic Vision has released all sorts of polls in Wisconsin in the last few years, all eagerly reported by the news media.
Now, all sorts of questions have been raised nationally about whether the firm even does polling, or simply releases numbers without making any phone calls. It’s not the first time those questions have been asked, but they are being taken much more seriously.
The American Association for Public Opinion Research (AAPOR) found that Strategic Vision …”repeatedly refused to release essential facts about polls it published prior to the 2008 presidential primaries in New Hampshire and Wisconsin. The AAPOR Executive Council announced today that this nondisclosure by Strategic Vision LLC was inconsistent with the association’s Code of Professional Ethics and Practices. and contrary to basic principles of scientific research.”
Details of Strategic Visions polls have long raised flags among pollsters, in part because it refuses — unlike other pollsters — to release “cross-tabs” — the detailed demographic breakdowns of individual polls. A source noted other anomalies to me today. One is that the pollster always reports having called a round number of respondents — unusual in an industry that typically uses large call centers and winds up — as casual poll readers know — with uneven numbers of calls.
Another question is how the firm pays for its polls. Its website lists at least 172 public polls, and at a stated cost of $30,000 a poll, that’s an expenditure of more than $5 million — quite a sum for a small firm.
A third question has to do with the firm’s offices. Its website, as recently as last month, listed offices in Atlanta, Madison, Seattle, and Tallahassee — all of which match the locations of UPS stores, rather than actual offices. The addresses are now gone from the site entirely, though it now also lists a Dallas presence.
Nate Silver, the numerical genius who emerged during the 2008 campaign, at his website, 538, says an analysis he’s run of expected results indicate:
One of the questions, in light of Strategic Vision LLC’s repeated failure to disclose even basic details about its polling methodology, is whether the firm is in fact conducting polling at all, or rather, is creating fake but plausible-looking results in order to increase traffic and attention to its core business as a PR and literary firm.
I posed that question largely as a hypothetical yesterday. But today, I pose it much more literally. Certain statistical properties of the results reported by Strategic Vision, LLC suggest, perhaps strongly, the possibility of fraud, although they certainly do not prove it and further investigation will be required.
In the interest of saying “I told you so,” a couple of entries from my old blog, The Xoff files:
20 Sep 2006 by Xoff: Strategic Vision, the Republican polling firm that many dems think cooks the books — or doesn’t even interview anybody — has new poll results that show Gov. Jim Doyle leading Congressman Mark Green 46%-42% in the governor’s race. Link
8 Mar 2006 by Xoff: Question: who’s paying for these? Strategic Vision isn’t doing it out of curiosity. knowing who’s paying the bill would help put it in some perspective. and, yes, the same is true of others that show up from time to time. Link
Friday, August 18, 2006 by xoff: Republicans are touting a Strategic Vision poll today showing the guv’s race a dead heat, finding that more believable than one done for WISC-TV which has Gov. Jim Doyle leading Congressman Mark Green 48-38. There is some doubt among Democrats that Strategic Vision even makes any phone calls, since no one has ever seen any cross-tabs. Having it show up 12 hours after a public poll that has Doyle 10 points up makes it even more suspect. It’s just a little too convenient. Link
Unfortunately, Wisconsin’s biggest newspaper, the Milwaukee Journal Sentinel, has relied heavily on Strategic vision as a source for polling numbers. A search for “strategic vision” in the JS archives turns up 81 stories. Craig Gilbert of the paper’s Washington bureau consistently used Strategic Vision numbers in stories about the 2008 presidential race.
In 2006, the week before the Doyle-Green election for governor, Gilbert wrote:
There have been at least 24 independent polls in the race for governor this year.
The good news for Democrat Jim Doyle is that he has trailed in only one.
The bad news is that his support has reached 50% — the comfort level for incumbents — in only one.
The bad news for readers is that the newspaper made no attempt to sort out or identify which of those 24 polls might be more credible than the others. It chose to simply report them all as equally valid, doing readers a real disservice.
As news coverage of the 2010 governor’s race begins, you probably won’t see Strategic vision in the mix. While that would be an improvement, what’s really needed are some standards for which polls get reported and which don’t. All polls are not created equal, and many others which get reported are amateurish and questionable, if not fraudulent.
But with shrinking newspaper staffs and budgets, don’t look for a lot of improvement in coverage. Let the buyers and the voters beware.