Talk
Around the World in 500 Days of Pandemic Misinformation
Invited Talk, USENIX Enigma, 2022 · February 1, 2022
44 slides · ~3,200 words · ~16 min
Cite this talk
@misc{kelley2022around,
title = {Around the World in 500 Days of Pandemic Misinformation},
author = {Kelley, Patrick Gage},
year = {2022},
howpublished = {Invited Talk, USENIX Enigma, 2022},
url = {https://pgk.io/talks/2022-talk-around-the-world-in-500-days-of-pandemic-misinformation/},
}Below is a lightly edited version of the talk I gave at
Dear reader, this slide was on the screen before I started the talk, while I was being introduced. It’s an actual PSA that the World Health Organization made and distributed. The design work is not my own. Obviously
Hi everybody. As Kate said, I’m Patrick Gage Kelley, a researcher with Google Trust and Safety.
This is a photograph of a 5G tower that was lit on fire in April of 2020.
These are the cables coming out of a second tower.
This is a third.
This is a fourth.
There are at least 100 documented attacks on 5G towers in the United Kingdom, the Netherlands, New Zealand, and Canada over the past two years. These are attacks on cell phone infrastructure — something most of us would generally spend zero time thinking about — largely motivated by a range of misinformation.
This is (mis)information that’s been widely spread, including the idea that 5G radiation causes COVID-19, and that 5G signals weaken the immune system, allowing the spread of COVID-19. There are other theories unrelated to COVID-19 — including 5G causing cancer, 5G being advanced surveillance or weapons technology,
or 5G signals killing birds. But birds aren’t real anyway.
Later, 5G would resurface in a set of vaccine-related stories, around the idea of microchips in the vaccine, which became something of an internet joke — like this example with Ryan Reynolds on his Instagram, receiving the vaccine with the caption finally got 5G. This post, as of today, has 3.8 million likes,
and over 27,000 comments — which I can not generally recommend that you read.
But back in April of 2020, in New York Times reporting on these very real burnt-out 5G towers, Adam Satariano and Davey Alba write: “…the 10 most popular 5G coronavirus conspiracy videos posted in March were viewed over 5.8 million times. Today, the conspiracy can be found on Facebook in over 30 countries, including Switzerland, Uruguay and Japan.”
Which points to a potentially — hopefully — unique moment. The coronavirus pandemic has reshaped our world: our personal interactions, shaking hands, traveling to see loved ones, the masks we’re all wearing in this room; societal-level expectations around public health, the economy, who’s an essential worker, and who around the globe has access to vaccines.
This unique moment also means that much of the world’s attention is focused similarly on a single story — a moment that has now lasted for almost two years.
And so, using the tools of public opinion polling, in partnership with Ipsos, we can start to understand how pervasive these ideas are across multiple countries. The results I’ll discuss today include respondents from 16 countries, shown on this map, with surveys beginning in May of 2020 and continuing until today.
We asked two simple questions. The first was focused on exposure — we would ask people about an informational statement, and they would respond and say:
- I’ve heard that many times,
- I’ve heard that at least once,
- I haven’t heard that, or
- I’m not sure if I’ve heard that.
And we asked a second question focused on belief, where respondents could tell us if they:
- strongly believe a statement,
- kind of believe it,
- if they’re not sure,
- don’t really believe it, or
- strongly don’t believe that statement.
So for example, a single respondent saw this statement:
5G phone networks use radiation that weakens the immune system and makes people more likely to get coronavirus
— and a single respondent might have said that they’ve heard that many times, or that they haven’t heard that at all; that they strongly believe it, or that they strongly don’t.
If we look at our results from the United Kingdom, again back in May of 2020, we see that 28% of respondents there had heard that many times, with another 49% saying they’ve heard it at least once, and just 23% saying they weren’t sure if they’d heard it or were sure that they hadn’t — showing just how pervasive this misinformation had been, particularly after the attacks were on the news.
Across the other countries we have responses from, only one other had more than 50% reporting they’d heard this information — that’s Italy — while the other 12 had far less. The lowest, Japan, reported only about 21% exposure to the idea that 5G weakens the immune system. We see these country differences are large, and we’re also monitoring how they’re changing — to have some insight into how we should respond to this misinformation.
But although the UK had widespread exposure to the purported immune-weakening radiation of 5G towers, there was not widespread belief that this statement was true — and that’s good. At the time, about 75% of UK-based respondents said they didn’t really or didn’t strongly believe this was true, or that they weren’t sure, with only about 10% believing — with just 3% of those strongly believing. In some ways this is also good: the vast majority do not believe that they are being weakened by 5G towers around them in their environment. But 10% with some belief is still significant — 10% of respondents is not just noise, it’s not just limitations with surveys. This is a real, but small minority held view.
And at the time, the UK was actually a bit better off than some other countries: 14% of Australian respondents, 13% of Canadian respondents, 19% in France, 19% in Italy — with, in France and Italy, both 6% strongly believing this. So this is one of our first takeaways: there are a lot of people who hold these beliefs, in some cases quite strongly — beliefs that a few years ago wouldn’t have really made sense, but have built on conspiracy theories, likely expanding a base of belief for years to come.
Now, you might say: Patrick, this is old — May of 2020 is ancient history, and all sense of time has been removed from our earth since the pandemic started. In May of 2020 my sourdough starter was still alive — rest in peace. In May of 2020 people cared about hydroxychloroquine and 5G towers, and we’re way beyond that; we’re busy with anti-vax narratives, we’re busy with election misinformation. To which I say: yes, and we’re going to go there now, with this grounding of how our survey worked, let’s look beyond 5G to other information statements we tested.
So we’ve seen 5G, and if we averaged across all the countries we’ve studied, we see about 13% belief — that’s both kind of and strongly. We also tested this statement — hydroxychloroquine is an effective treatment for people without serious coronavirus symptoms — that had 19% belief.
We also asked: Bill Gates, George Soros, or some other powerful person is behind the coronavirus — that had 16% belief.
And we asked: injecting cleaning products or UV light into people is an effective treatment for coronavirus — that has 11% belief.
Additionally, we don’t just have one-time numbers. We’ve been running these surveys semi-regularly since May of 2020 — every few months, checking in to see how things have changed, or not changed. And this is still ongoing; we’ll have more data coming in over the next few months. One of the effects we find over and over again is that although the narratives move quickly — although you all have potentially not been thinking about injecting UV light or bleach recently — once these fringe beliefs take hold, they seem to stick.
Just about a year later, in spring 2021, we again asked respondents about those same statements and we found that that UK group had only decreased 1% — a non-statistically-significant change — which is reflective of what we saw in other countries. These beliefs had stuck. And this isn’t just true for 5G: when we look across differences from 2020 to 2021, we see relatively little change. Hydroxychloroquine as an effective treatment was unchanged. The idea that Bill Gates, George Soros, or someone else was behind coronavirus actually increased 1% year over year. And injecting cleaning products or UV light didn’t change either.
These views, once they are established, seem to stick. Which brings us to vaccines. Even before vaccines were available, it was much discussed in information-quality circles that vaccine misinformation would be coming, and it would be big — again building on multiple decades of anti-vaccination community organizing.
We tested multiple items on coronavirus vaccinations, including: the coronavirus vaccine has microchips in it to secretly track those who get vaccinated — which is currently at 11%; it decreased just 1% year over year.
The coronavirus vaccine may cause infertility, so people who might want to have children in the future shouldn’t get it — that increased 2% since we first tested it.
When a coronavirus vaccine is available it will likely harm many of the people who get it — that increased 5% from when we first tested it, up to 25% last summer. That means that across a time where millions of people first safely received the vaccine, we saw increasing mistrust in vaccines that might harm people. Again, we see relatively little change in these numbers.
While we focused this talk around measuring and understanding COVID-related misinformation, it’s important to point out that not all misinformation is created equally — you’ll get a better sense of that in Gus’s talk next; and you heard a bit about this from Chris in terms of disinformation. I’m not going to get into separating out disinformation from misinformation, the intent, and what qualifies as this either.
Partially, because we’ve seen information shift throughout the pandemic — a difficulty is a changing information landscape. When we first asked about hydroxychloroquine, it was arguably still up in the air — there were ongoing trials, and while things were starting to not look great, it wasn’t deeply down the very polarized, confused path that it would then follow, one that — say — ivermectin, convalescent plasma, remdesivir, monoclonal antibodies, and others would later follow. And so from the beginning we included statements that were not purely misinformation not just conspiracy theories.
As an example of not just testing mis/disinformation, we asked participants if they had heard or believed that a coronavirus vaccine supported by scientists and medical doctors around the world would be safe. We first tested this question in October of 2020 — before vaccines were available, before phase-3 results had come back from any of the major companies — and at the time only 52% of respondents believed this: believed that a vaccine supported by scientists and medical doctors would be safe. That went up 4% year over year, to where it was sitting last summer. You may find 56% to still be quite low for a vaccine that the medical community supports.
We also tested some general health information — and, if we are looking for optimism, perhaps this is where we focus. Overall, we see much higher rates of belief than any of the more clear-cut misinformation or questionable-information statements that we tested.
We asked about:
Wearing a cloth face covering or mask in public is an effective way of slowing the spread of the coronavirus,
and staying at least six feet from people not in your household is effective in slowing the spread of the coronavirus — are two of the questions we tested.
In mid-2020 we saw these numbers at 73% and 70% strong belief, strong support, across the 16 countries we tested these things in. This is great — two of the highest agreements of any item we tested.
However, these were also two of the items where we saw the largest drop between 2020 and 2021 — 5 and 7 points respectively. And while this keeps both in the 60% agreement range, it shows how much effort is required to maintain these extremely high levels of support. How important continued, unified, proactive health messaging is.
And we see this in specific countries. If we look at just the numbers in the United States, between May of 2020 and May of 2021 these items both drop by 8% — likely capturing some of the polarization that happened over the course of this year, a change in leadership, and a change in messaging.
We see specific in-country effects for some of these items. In China, belief that social distancing was effective — which started around 71% — dropped over 15%. This likely is related to China’s radically different approach to COVID policy: how they support lockdowns, extremely strict lockdowns, regular-testing, and vaccine requirements.
I want to pause to reflect more deeply on how in-country differences determine how we need to look at these numbers. We had motivated our survey, saying the entire world is similarly focused on this one event, which is true. But we see that local policy, local decisions, and the ways this interacts with people’s daily lives impact how they believe and how they are exposed to this information.
Even without the time to explore every country’s individual results today, the entire world focused on the same general problem doesn’t mean that misinformation is simply an overlay that affects every country and its people in the same way. From different national and government responses, to the impacts felt by different regions, to the availability of vaccines and other societal factors — these all determine how messages are spread, heard, and believed.
Additionally, all the standard methodological limitations apply. These survey questions were written by a United-States-based research team with western views. While we worked with our research partners to translate and localize the results, and we do our best to acquire the best panel responses that we can, there are still difficulties here. For a more complete description of these issues, see my Enigma talk from last year on multi-year opinion surveys of privacy attitudes — back when my hair was still pandemic blonde.
While we’re focusing on the limits of this work — we also have just two metrics: exposure and belief. But what’s the relationship between these two? If we think of one’s belief in a piece of low-quality or potential piece of misinformation — an obvious one, perhaps: say that powerful people are behind the coronavirus — how does your self-reported exposure relate to your self-reported belief in this item?
If we take the group of people who reported that they weren’t sure if they had heard this as our 1x baseline, then if you self-report that you’ve heard this many times, you’re 7.9x (7.9 times) more likely to believe this is true. That is: a member of the group who self-reports that they’ve heard this many times is 7.9x more likely to believe that some powerful person is behind coronavirus than a member of the group who reported that they aren’t sure if they’ve heard this, or that they haven’t heard it.
This isn’t just for this example. We see that likelihood of belief is 6.2x for the 5G-signal statement, 6.3x for general vaccines being harmful, 7.2x for bleach or UV light as an effective treatment, and 11.7x more likely to believe that COVID was created as a bioweapon, if you’re someone who says you’ve heard this many times. Overall, these odds ratios tell a story: that self-reported multiple exposures has a relationship with belief.
But this data alone, again, can’t tell the whole story. We don’t have enough information from this survey to explain anything causally — I can’t say that just being exposed to this information multiple times will lead to belief. Possibly belief forms, and then people go read more and more about some of these theories. We also don’t know where and how these exposures happen: people have potentially self-selected into groups where this information could be shared regularly, and the choices that led there may also lead to their beliefs. In this survey we don’t know if our respondents were proactively trying to find this information; we don’t know what news sources or real-life conversations or other events brought them here. And we aren’t capturing in this survey what these beliefs led to in terms of real-world outcomes — for example, we don’t know which of our respondents went and talked to others about the dangers of 5G or actually set fire to 5G towers, though likely none of them in this sample.
One real-world outcome we do have from the survey — again, self-reported — is who got vaccinated in the later waves of our data, when vaccination was becoming more available. We see that participants who believe that vaccines are generally unsafe are half as likely to get vaccinated, and participants who believe that when scientists and doctors say a vaccine will be safe, it will be, are about twice as likely to get vaccinated. This isn’t surprising — that vaccine beliefs would lead to vaccine action — but we’re really trying to say that these beliefs, this exposure, does have meaningful real-world outcomes. It’s not just about cell phone infrastructure; it is about public health.
So to conclude, I want to again highlight a few of the things we’ve learned from surveying respondents in 16 countries over the last two years.
We see substantial minority populations in every country believing in various misinformation and low-quality-information statements, as well as widespread exposure to that information. The reason we continue to do this public opinion polling work is that it’s one way to better understand and measure misinformation — it’s one signal out of many — and it highlights differences between countries that deserve further investigation.
We also see that fringe beliefs, sometimes strongly held, mean we need to take these types of beliefs in coordination with other things we’re thinking about — including non-pandemic topics. There are many beliefs held by these individuals, sometimes strongly, and this is something we need to deal with as a security, privacy, and safety community, and as a broader society — because once these beliefs take root, they stick. So understanding how to stop this in the first place — pre-bunking, as was mentioned earlier, or other ways of limiting the spread of this exposure and this belief — may help us. We see information that has left the news cycle still holding similar levels of belief to surveys from over a year ago, and, as always, we see really slow change in these public opinion.
And finally, we see a relationship between exposure and belief — and while this survey can’t determine causality, we do see that people who report seeing certain information more often also report higher levels of aligned beliefs.
We think it’s vitally important to understand the quality of information that exists on and offline, and how that information leads to beliefs and behavior — in this case, behaviors that could support or harm online safety and public health. And we’re continuing to monitor these items as the pandemic continues.
Thank you all so much — I’d love to take questions.
Final slide just for the aesthetics.