Polling has become harder (and faker) than ever · ↗ www.theargumentmag.com

Charlotte Swasey has a good piece over on The Argument on how difficult it has gotten to accurately poll the public (and how easy it has become to poison the discourse with dubious or outright fake polls).

In the golden age of polling—maybe the late 80s to the early 90s—when everyone had landlines and most people would pick up the phone, polling was pretty easy to do: just randomly dial numbers and survey a random adult in the household. Sure, it was labour-intensive, and you still had non-response bias. But for the most part, you could pay people a bunch of money to make a lot of calls, then make a few straightforward adjustments for who responded and assume your resulting survey was relatively representative of the population you were trying to make inferences about.

None of that is true today.

While it has gotten easier and cheaper than ever to survey the public through any number of firms offering online panels, actually interpreting these results is extremely difficult because respondents are extremely self-selected. At the same time, this reduced barrier to entry has made it extremely easy for activist organizations to produce favourable-but-dubious polls to drum up headlines or mollify their donors.

Companies are even releasing fully AI-generated survey samples, often without clearly labelling that the results are purely fabricated.

Of course, the incentives to make bad polls are stronger than ever. Random polls tweeted out by poll-aggregating accounts with minimal vetting can shape narratives or even move prediction market odds (I wrote about the attention received by a poll from a fake polling firm last month in the context of the Los Angeles mayoral election). Because prediction markets move on new polling, there is also a financial incentive to create juicy headlines. (It doesn’t help that the prediction markets themselves regularly tweet out fake or rage-bait headlines.)

For now, putting out bad or outright fake polls is basically without consequence. Until we start sanctioning the creation and promotion of these polls, formally or informally, we should expect to see more and more of them.