Interview Research and the Prior Conclusion

justin-katz-avatar-smiling

A few days ago, the Internet handed around Molly Ball’s essay in The Atlantic about a Democrat think tank’s “safari” to observe residents of the flyover states in their native habitat.  The temptation on the right has been to take the lead of the essay’s title and focus on the notion of treating one’s fellow Americans as a strange, distant tribe of scarcely recognizable foreigners.

On that theme, one point worth pausing on is its reinforcement of federalism.  Progressives generally don’t mind leaving strange tribes strange when they’re far away, so why would we expect one detailed national policy regime to be appropriate for our entire large country?  (One suspects the answer has something to do with wanting the power and money that goes with such control.  Benighted tribes have more value as novelty displays.)

However, Ball’s intended emphasis is worth considering, as well:

That moment of doubt does not appear in the report that Third Way released, which distills the group’s conclusions from the tour I joined. In the report, there is only one quotation from the hippie roundtable in Viroqua—a man who extols the area’s turnaround, in a section about the area’s “intense local pride.” “There’s love, beauty, and a sense of opportunity,” he is quoted as saying. “There’s been a rejuvenation of identity.”

In the moment, [researcher Nancy] Hale had heard sentiments like this as disturbing, of a piece with the community’s self-satisfied separatism. In the report, it had been made to sound like a paean to localism.

The report surprised me when I read it. Despite the great variety of views the researchers and I had heard on our tour, the report had somehow reached the conclusion that Wisconsinites wanted consensus, moderation, and pragmatism—just like Third Way[, the think tank doing the research]. We had heard people blame each other for their own difficulties, take refuge in tribalism, and appeal to extremes. But the report mentioned little of that. Instead it described the prevailing attitude as “an intense work ethic that binds the community together and helps it adapt to change.”

To Our Readers: We need your support to challenge the progressive mainstream media narrative. Your donation helps us deliver the truth to Rhode Islanders. Please give now.

We should take Ball’s essay as a reminder to be skeptical about studies that purport to summarize public opinion as gleaned from interviews.  Where data can be reviewed, one can be more trusting of research; where it amounts to interpreting a variety of conversations to which readers have no access, not so much.

This goes, as well, for activist and government efforts like the Rhode Island Senate’s round of PawSox stadium hearings or the events government sometimes organizes in order to “gather information” in the service of policies like RhodeMap a few years ago.  The starting expectation should be that their actual purpose is to advance a prior conclusion, perhaps allowing participants to fill in some unimportant details for buy-in.

Click to help us keep the doors open.



  • Rhett Hardwick

    I recall reading (but cannot verify) that Obama people assigned to “work” NASCAR events demanded inoculations against the various diseases they were sure they would encounter. If they did interviews, what conclusions do you think would have been reached?

    Those few politicians I have known indicated that “input” and “comment periods” were designed to wear down the opposition before they did what they wanted after “taking input”.

Quantcast