Peeking at the answers

PZ Myers spots a new nationwide web survey from the Christian Coalition of America.

The Christian Coalition of America has created something called the 2008 American Values Survey. It is why we crash polls.

This particular poll is supposed to produce results that the Christian Coalition will present to congress as if it is a serious and representative sample of “American values” … which must be why they toss it out onto the web on a conservative website. This is not how you do a legitimate poll. This is how you bias your results.

So let’s all show them that American values include atheists and agnostics and humanists. More importantly, churn this badly designed poll so that either the results are far more ambiguous than they hope, or they have to start throwing out answers they don’t like selectively, further demonstrating the invalid nature of this method.

Web surveys are usually nearly useless, as the sample taking them is self-selected (only people who want to take the survey, who are motivated to give their opinions, do so, which means it’s not a representative sample) and usually biased by location (if you put a survey on Daily Kos, it will likely lean left; if you do it on the CCA page, it will likely lean right). However, as Myers notes, it’s not the first time that such a biased survey has been presented as the model of fairness and the Overwhelming Voice of the People.

I don’t worry too much about Congress being taken in, since any Congresscritter worth his election polls knows just how such surveys are goofy. However, if presented to the American public as a whole — almost certainly as a counter to the National Survey we had on 4 November — it might confuse some people about what the Average American really thinks (since the survey will not be of average Americans).

I hopped through to the above link and initially had problems getting through, coming up with an odd system error. Later, it came up to a blank page. I poked around and found the current web address of the survey. I suspect some early articles like Myers’ have already had their effect, hence the move (and, I suppose, reboot).

The text of the survey intro is a bit different than Myers characterizes it above — it’s not clear if that’s changed, or if the results would actually be presented with the same preamble:

Given the changes in Washington, it is urgent that political leaders understand that America’s Christian voters expect strong moral leadership in government. Leadership that respects and reflects our values – not attacks them at every opportunity.

That’s why we’re conducting a massive survey of America’s politically active Christians – so we can show the politicians and the media exactly where we stand.

We’re surveying hundreds of thousands of conservative Christians nationwide, and we will announce the results in Washington, DC in January when the new Congress is seated.

This is your chance to send a strong message to America’s political leaders as well as members of the media. Urge them not to ignore the issues and concerns of America’s Christians. Don’t let them claim that they never heard from you.

So now it’s presented as a way for “America’s politically active Christians” (which, I hasten to note, ahem, is not the same as “conservative Christians nationwide”) to have their voices heard. Arguably, of course, they had such a chance back at the beginning of the month, but …

The survey itself is something of a hoot, with both standard talking points and “push poll” style questions that are framed to elicit the answer they are looking for. In many cases, the “values” involved are rather vague and/or simplistic. “Should Congress grant amnesty to illegal aliens living in our country?: ” comes in the “constitutional liberties” section, even though it has only a sketch relationship to constitutional liberties, let alone Christian values; further, of course, it’s a simplistic question to a highly complex issue.

Indeed, there are a lot of questions that strike me as much more economic and legalistic than religious. What do private retirement accounts (or alternative energy policy), I might ask, have to do with Christian (or American) values (at least as usually presented by groups like the CCA)?

Anyway, I took the survey, though I suspect they may not like some of my answers. We’ll see what comes of it