The observer effect refers to the fact that merely observing or measuring some parameter will change that parameter and Religion Dispatches has a great interview with Robert Wuthnow, professor and chair of the Department of Sociology at Princeton University on Jimmy Carter, the “Nones,” and poll data about religion where he outlines how pollsters have actively sought to manage our understanding of religion in modern culture.
Here’s a sample:
One is that a number of the people who are categorized as Nones still claim to believe in God. Many of them occasionally attend religious services; hardly any of them identify clearly as atheist. So they may be, for some reason, identifying themselves as non-religious even though they still believe. We also know from some of the surveys that they identify themselves as “spiritual but not religious,” meaning that somehow they’re interested in God and spirituality, and existential questions of life and death, but are turned off by organized religion.
Second, the political climate over the last ten to fifteen years appears to be part of the story. The religious right has become so publicly identified with conservative politics, that people who were formerly willing to say, “I’m religious” in [denominational] terms are now saying, “You know what, I just don’t want to have anything to do with it.” That’s a hard story for many of us to believe, but there does seem to be pretty good data supporting it, as part of the story at least.
Then, number three, is that the studies that actually ask a person the same question a year or two later are finding that individuals change their minds a lot. That seems to occur especially with people who are identified as Nones. One paper identifies at least half of the Nones as “Liminals,” people who are trying to decide, on the cusp of making up their mind. You ask them one year, and they say “I’m non-religious,” you ask them the same question the next year and they’ll say they’re religious and they’ll tell you what kind of religion they are. Or [vice-versa].
And in thinking about how we should interpret religious polling;
Well, first and foremost, remember that the poll numbers being reported may be in the general ballpark correct, but probably can’t be interpreted very precisely in terms of the small trends that are being reported. For instance, if somebody reports that American religion is declining because church attendance is down a percentage point this year from last year, don’t pay much attention to that.
Secondly, pay attention, skeptically, to the way the headline describes the data. So once again, let’s imagine that the church attendance rate is lower this year than last year, let’s say it’s two or three percent lower all of a sudden. Does the headline say that religion is “on the skids” or that “America is losing its faith”? That’s more the journalist’s or the editor’s fault than the pollster’s fault. Polling usually doesn’t produce news. It usually just produces possibly interesting information—it isn’t news in the sense that something really happened, and so editors and reporters have to think of some way to make it seem like news.
Third, always remember that the polls have a very low response rate. Most of the polls, whether about religion or politics, have an eight percent response rate now. It means that ninety-two percent of the people who should have been contacted for it to be a representative poll are not there, and we don’t know what they would have said, and so we’re only making guesses. The guesses could be wrong, and often are wrong.
Is polling a useful tool in the church’s mission or not? What do you think? How have you seen it used?