Misinformation Discourse, Cognitive Science, and Independent SAGE

A post orthogonal to a book review of *How to Expect the Unexpected: The Science of Making Predictions and the Art of Knowing When Not To*, by Kit Yates

When I became a mom, I thought I had done my homework. In the case of infant feeding, this meant reading up on breastfeeding in standard books along with peer-reviewed articles on PubMed. Then, through a combination of bad medical care and fixed belief in what I thought I knew based on this education, I accidentally starved my son for over a month under the auspices of “exclusive breastfeeding.” In addition to producing some interesting documents via the Freedom of Information Act (FOIA), along with a few peer-reviewed articles on how common and serious this type of problem is, this experience eventually led me to begin thinking more critically about science and science communication more broadly.

Part of what had been so powerful in my own experience of believing myths was the rhetoric of myth-busting that exclusive breastfeeding proponents often use. (See some of my other work myth-busting this myth-busting here.) It was hard to question the myths that I had learned (wrongly) corrected other myths. Especially when the purveyors of those myths seemed to be citing scientific chapters and verses.

This sort of difficulty poses an important challenge for science and society that goes far beyond infant feeding, itself an issue that affects parents and children worldwide. For science, the challenge is incorporating learning — letting better methods replace worse ones (as quickly as possible), admitting mistakes, and applying the lessons of science, to science. This is about the causal revolution, open science movement, and statistics reform, especially surrounding the need for cognitive science in methodology. For society, the challenge is losing the secular religion of Science, coming to terms with our fallibility and non-neutrality problems, and still managing to build enough trust in public institutions that we can tackle emergencies like pandemics and the climate in reasonably reasonable ways. Both challenges require accepting uncertainty and practicing humility — not noted strengths of humanity as a species or our cultural moment in particular.

This post proceeds as follows. First, I define two sorts of scientific projects one could do with information science — synthetic-directive and critical-reflective. Then, I suggest it makes most sense to do them together. Next, I point out a recent addition to the common phenomenon of scientists doing the first and not the second — Kit Yates’s How the Expect the Unexpected: The Science of Making Predictions and the Art of Knowing When Not To, published in July 2023. The book’s political and social context is notable for its author’s position in the Covid discourse: Yates is a University of Bath Senior Lecturer in Mathematical Sciences, co-director of its Centre for Mathematical Biology, and a leading epidemic modeler appointed to the UK’s Independent SAGE, a scientist advisory group that has championed stricter Covid policies. He’s also a prominent science communicator.

Reviewing a few of Indie SAGE’s myth-busting pronouncements, I show that this corner of the Covid misinformation discourse is itself rife with misinformation. This is ironic (i.e., contrary to what is expected) — if you expect self-described myth-busting to actually bust myths. But if, on the contrary, you expect people to be fallible and biased by our own perspectives, even when we’re scientists who think we’ve got it right — then it’s expected. So, in this case, lowering your expectations to recognize that science is done by human beings might seem like a good first step in expecting the unexpected.

A Tale of Two (Possible) Projects

There is a synthetic project to be done in the big backyard of science as a whole — statistics, information science, data science, research methods, or however you want to conceptualize it. That project involves putting together what we know about risk assessment, cognitive bias, and heuristics to help people learn to make better decisions. It’s directive, because its aim is to produce knowledge about what people should do. So synthetic-directive is one flavor of this type of big-backyard scientific project. Most science communication does this.

And then there is a critical project to be done, wedding methodological discourse on problems in science with the science communication imperative to help people make better decisions given the (sometimes neglected) limits and context of science itself. But because the critical nature of this project is self-aware — we are human beings vulnerable to bias, doing science, and so our science is vulnerable to bias — it is more reflective than directive in its pronouncements. So critical-reflective is another type of big-backyard scientific project. It’s rarer.

It doesn’t make sense to do the first type of project without also incorporating elements of the second. That would be internally inconsistent, because scientific methods are largely concerned with getting closer to the truth by struggling against various biases, and critical-reflective science (meta-science) is about doing the same thing in science. But Yates’s How the Expect the Unexpected takes on the former task, ignoring the latter.

That focus produces preaching from the pulpit of Science that rings, at best, naive. For instance, Yates speaks admiringly of scientists “Making predictions that were later falsified or validated by experiments.” (p. 398) But he omits the fact that this happens only in a small proportion of studies, often still subject to debate and modified understanding through further research. In reality, most scientific evidence contains multiple dimensions of uncertainty and ambiguity requiring interpretation. Meanwhile, most of the questions worth answering, can’t be answered with experiments for both practical and ethical reasons.

Ignoring science’s ongoing struggles with these limitations and uncertainties, in Yates’s narrative, knowledge production “was crucial to driving knowledge forwards… As our scientific knowledge and practices improved, we were able to make better, more reliable and general predictions.” This is a linear arc of scientific progress. It conspicuously omits mention of the replication crisis, and other widespread concerns about just how credible and cumulative most scientific knowledge production really is.

Yates continues: “Science also provided us with the tools to expose those unscrupulous characters who were out to exploit us with their unfounded rhetoric.” In this narrative, science rescues people from exploitation by misguided, corrupt forces. Those forces apparently never include scientists or scientific institutions themselves.

Science does this by helping us fight biases. Biases that How to Expect details, albeit neither comprehensively (Wikipedia may do that best), nor in a format that helped me better organize my own overview.

But these biases are universal. So there’s a regress problem. Using special scientific software, I was able to model this technical representation of the problem:

Why Science Needs Meta-Science

The problem with this type of “romantic heroic-fantasy science,” as leading statistics reformer Sander Greenland calls it is, is that “There’s Not Much Science in Science” (video/slides). So this book is a social document of a contemporary view of science without the critical lens of meta-science. Written by a scientist who participates in creating this social entity we call science, whose areas include statistics and cognitive bias, and whose work affects millions of people’s lives.

This book’s project is important. Indie SAGE’s work is important. But it is incumbent on Yates and other scientists doing these sorts of jobs to also try to bring to bear some awareness of how cognitive bias commonly affect scientists and statistics. At least acknowledge the problem. Otherwise it’s not safe to do science!

Because science without meta-science is like driving without a seatbelt: It makes the possible consequences of your mistakes, or someone else’s, much worse, for no payoff. It’s a bad social norm in the practice of science that scientists should change. One reason it doesn’t change is probably that structural conditions like information overload and increasing precarity make everyone too tired and busy to think properly. Everyone except the rogue methodologist ninjas.

Another reason science can seem curiously lacking in meta-cognition is probably that people tend to dislike admitting mistakes and uncertainty. So it is difficult to see how to make criticisms like this in a way that their subjects will be likely to hear them out. No one likes a critic.

But I tend to think of meta-cognition about the social and political creation and consequences of science as more game than formula. There’s always a next level to play. Putting a ludic spin on the thing is at least one possible coping strategy for the harsh reality that we cannot escape our fallibility and non-neutrality (or others’).

Misinformation Misinformation Damnation

Here’s what this critical awareness might look like, applying some standard statistics and cognitive science fare (the stuff of Yates’s book) to Independent SAGE’s work. A few disclaimers…

Disclaimer 1: I have not followed Independent SAGE closely. My prior is that they’re credible, well-intentioned, top-notch scientists doing important public health work in the public interest during a time of terrible stress. Thank you, Independent SAGE!

Disclaimer 3: See what I did there? I made a numbering mistake! Idiot. Seemed like a good way, though, to acknowledge that I’m caught in the same fallibility and non-neutrality traps as everybody else.

There should be a higher bar for acknowledging and ideally also addressing this problem in misinformation research, because otherwise its purveyors are engaging in explicitly false advertising (“get your corrected information here!”) as compared to regular research (“get your information here”!). Both discourses could be corrected with a disclaimer; something like “terms and conditions: regular fallibility and non-neutrality apply.” We would then, however, simply learn to read over that disclaimer.

Better, then, if scientific methods incorporate checks for mistakes and biases, do a bit of self-reflection identifying them, or even model what different assumptions would mean and outline the resultant uncertainties. There are lots of options here. Near-total lack of recognition of the problem — the default option in science and science communication as they’re currently, normally practiced — is not among the better options.

This is ironic. Experts who call themselves myth-busters — like some who’ve published on abortion and mental health, exclusive breastfeeding (see my previous work debunking this here), and Covid “misinformation”— need to be especially careful to check the bias and error they claim to be correcting, but are likely correcting imperfectly. Because “myth-busting” is a political designation staking a truth claim against an imagined, lying Other.

It’s almost as if there is a taboo against doing this. Could it be because myth-busting is a moral high ground, and it feels so good to take it? Or because performing certainty is seen as the more professional presentation of doing science as social reality? I’ve been guilty at times of doing both of those things, myself.

No one (almost) wants to say we’re all caught in an infinite regress of subjectively observing, analyzing, and interpreting data on pretty much every important human outcome. But they can’t take away your scientist card if you do. To the contrary, it serves science and society better to admit this, because otherwise the accessibility of digital information can net erode trust in public institutions like the WHO when it becomes obvious they make mistakes (plenty) — instead of the discovery of such mistakes looking like an inescapable part of a valid, ongoing process.

Disclaimer 4: I had to start somewhere, so I made my own shortlist of “Things (Almost) Everyone Gets Wrong About Covid,” and Google searched on “Independent SAGE” and that list. My list (ordered by cognitive accessibility/the order I thought of it in). This was not an ideal protocol for a systematic review; but this is not a systematic review. My list:

1. Ivermectin

2. Young male vaccine myocartitis

3. Mandatory vaccine boosters

4. Long COVID numbers

5. Death counts

6. Reproductive effects

Then I followed the search results from IndependentSAGE.org down enough rabbit holes to make a post SubStack started displaying a banner warning me was “too long for email” (my usual post length). That left a lot of rabbit holes. The point, again, wasn’t to do a systematic review of Independent SAGE’s recommendations, or even to work through this list; but just to look at a few cases where they probably got it wrong, and think about the cognitive biases and power that these mistakes reflect. Because there is not a surplus of critical work on misinformation discourse that works not on the plane of us versus them, but rather on the plane of seeing the stitching between the interpretation and presentation of scientific evidence as neutral, certain reality — where power has done the stitching, and bias, error, ambiguity, and uncertainty are sticking out with the stuffing.

Example 1: Ivermectin

Indie SAGE Argument

Independent SAGE member and University of Manchester Professor in Biomedical Sciences and Public Engagement Sheena Cruickshank recently noticed Ivermectin trending on Twitter, and in response promoted fellow Independent SAGE member and University of Leeds Professor of Cancer Virology Stephen Griffin’s Youtube video, “Myth-buster 6: Ivermectin and Hydroxychloroquine are effective treatments for Covid?”

In the video, Professor Griffin says (my paraphrase): At the beginning of the pandemic, we tried using existing drugs against the virus, and some promising results probably showed these drugs affecting the cells in which it’s grown.

That’s a hypothesis, and I’m not sure what he means. Is this growth in vivo or in vitro? Wouldn’t we care about in vivo inhibition of viral replication? If it’s only in vitro, wouldn’t we expect to see no possible benefit in human trials if it was just a lab artefact?

He continues:

So unfortunately, with this, there became a conspiracy around it that these cheap drugs that are effective for the things they’re supposed to treat. So, hydroxychloroquine is used in things like lupus. Ivermectin is used to treat parasites. There seemed to be a conspiracy that built up around this saying that the drug companies are trying to make more money by developing more drugs to sell to us instead of using these very cheap ones, which they couldn’t make money from. Unfortunately, that really isn’t the case. And in fact, Merck, who of course make new drugs, actually, the people that manufacture ivermectin, so they would have stood to make a lot of money had it been used against SARS-CoV-2 and Covid. [Bold and italics mine.]

There are a few problems here:

1. Calling something a conspiracy as a way to discredit it is the subject of a famous George Carlin bit. Carlin’s point is that this designation is a political act with political consequences. Here, it’s a political interpretation of a possibility that we can examine empirically.

2. We don’t need that political interpretation to entertain the empirical possibility that ivermectin might help in Covid.

3. Griffin’s argument doesn’t make sense as a political interpretation, either. Apparently, “Merck’s patent on ivermectin expired in 1996.” There are generics. Not that Griffin alleges the “conspiracy” was rational. But his defense against the “conspiracy” involved attacking its rationality; and that defense seems irrational.

Griffin continues:

“Now, the problem is that we now know from multiple, very large, randomised controlled clinical trials against placebo, that neither of these drugs actually help in Covid, whether it’s given early, whether it’s given before you’re infected, whether it’s given when you’re severely unwell. These do nothing, I’m afraid. And we have robust trials that show all of this. Please ignore all of the nonsense that have been posted on Twitter by former US presidents etc. Please ignore some of the doctors’ associations who spout this this is actually going to help you with Covid. This is not true. [Bold mine.]

In the case of ivermectin, this position is wrong. Credit for this argument but no blame for my possible mistakes goes to Sander Greenland and colleagues, who have been using ivermectin research misinterpretation as a keystone case study in significance testing misuse for a few years now. For the full academic version, see “To curb research misreporting, replace significance and confidence by compatibility: A Preventive Medicine Golden Jubilee article,” by Sander Greenland, Mohammad Ali Mansournia, and Michael Joffe, Preventive Medicine, Volume 164, November 2022, 107-127.

Greenland et al Critique

In the above-referenced talk “There’s Not Much Science in Science,” Greenland reiterates (p. 42):

“Ivermectin to prevent hospitalizations in patients with COVID-19’ BMC ID 2 July 2021…

• Abstract: OR = 0.65; 95% CI 0.32, 1.31; p = .23 reported as ‘Ivermectin had no significant effect on preventing hospitalization.”

• Medical news sources reported that the trial ‘found no benefit for ivermectin on death” —

• BUT on p. 5 of the paper: OR = 1.34, 95% CI 0.30, 6.07 from 4 ivermectin, 3 placebo deaths.

• The trial was too small to show anything!

In other words, ivermectin may confer a benefit in preventing Covid hospitalization, or not. To say instead that it definitely does nothing, as Griffin does, commits the “absence of evidence = evidence of absence” logical fallacy that is rampant in epidemiology and medical statistics. The 95% compatability (aka confidence) interval shows absence of definitive evidence here. But that’s not evidence of no effect; it’s evidence of uncertainty about a possible effect. Uncertainty that’s entirely absent from Griffin’s (mis)representation of the facts.

The same is true of pre-exposure prophylaxis with hydroxychloroquine, with Greenland noting of García-Albéniz et al’s Eur J Epid 2022 “Systematic review and meta-analysis of randomized trials of hydroxychloroquine for the prevention of COVID-19”:

Pooled RR estimate from 7 pre-exposure prophylaxis trials: 0.72 (0.55, 0.95) [4 post- exposure trials: 0.91 (0.62, 1.35)], cautiously reported as “A benefit cannot be ruled out based on the available evidence from these trials”… Similar observations apply for ivermectin.

In other words, evidence on ivermectin and hydroxychloroquine shows possible benefit, but has been widely panned as showing abject failure. Greenland contrasts that consensus interpretation of and establishment response to these sorts of results, with the interpretation of and response to trials such as the PANORAMIC trial for molnupiravir, a new drug that’s now-standard in Covid treatment and costs a lot of money. The empirical point is that Griffin is wrong: Evidence suggests ivermectin and hydroxychloroquine may have beneficial uses against Covid. The evidence contains this possibility. It does not contain the certainty he communicates.

The larger, political point is that painting people who disagree with your read of the evidence as crazies is a cheap trick. As it happens, it does appear that Merck made a lot more money off molnupiravir than it would have off of ivermectin as a Covid treatment. And that they’ve continued to profit in this way (2022, 2023).

Recap: No exit from stupidity

The point here is not that Greenland’s conspiracy theory is better than Griffin’s (although it is). They are playing different games. Griffin/Indie SAGE are claiming to be doing work under designations like “myth-busting” and “fact-checking.” Greenland is claiming scientists need to practice more evidence-based self-awareness of the fact that we’re all imperfect fish living inescapably in the same cultural water (and mud).

This need for cognitive science in science implies that misinformation discourse like Indie SAGE’s could stand to have a little more humility. As Greenland notes, leading methodology reformer and Columbia University professor of political science and statistics Andrew Gelman makes the same point about behavioral econ “nudgelords.” Griffin’s contrastive slurs like “misinformation,” “myth,” “conspiracy,” and “nonsense” that others “spout” should perhaps be used, at a minimum, with recognition of the fact that using this terminology is political.

Misinformation discourse like this, then, is a social document. What social purpose might this sort of language serve? My sense is that it delineates an US (following science, debunking myths; good, noble, honest, public-serving) versus a THEM (spouting nonsense, conspiracy theorists; wrong, corrupt, misleading). This sort of black-and-white opposition is central to authoritarianism, prejudice against outgroups, intolerance of the Other, and all the rest of the associated social and political troubles that good left-liberal/right-centrist scientist types tend to think of themselves as being above. But Covid is a threat. Threat tends to make us more authoritarian, regardless of political persuasion; “fear is the mind killer” and all that.

Maybe I’m wrong, and there’s a conspiracy to tell people “noble lies” that ivermectin and hydroxychloroquine have proven useless against Covid when they have not, because these meds are limited and needed for other uses, like against autoimmunity and parasites.

But probably, there is just sufficient incompetence in interpreting statistical evidence at a sufficiently high level that drug companies like Merck really did make huge profits off new, expensive, and possibly ineffective or minimally effective drugs, while old ones were comparatively cheap and possibly just as (in)effective. Meanwhile, establishment propagandists unwittingly covered (and continue covering) for this actual possible conspiracy by repeating bad science, or the bad science communication of it that they got from someone else, in the telephone game of interpretations that is knowledge as a social production. Again, the point is not that Greenland et al’s interpretation is better than Griffin’s (although it is). Rather, the point is that science tends to have a lot of ambiguity and uncertainty that requires interpretation. That interpretation is subject to the same sorts of cognitive biases to which all other human pursuits are subject.

We can correct them! We should correct them! But we will still be imperfect. And it will help, in doing better science, if we practice more self-awareness of the need to do this correction in the first place, by recognizing the need for cognitive science in methods. The first people to recognize this should be the people who write books about things like cognitive bias in decision-making.

Which cognitive biases seem to be at work here? With significance testing misuse, Greenland’s stock criticism is of dichotomania (reading dichotomy into more nuanced reality). This gets back to uncertainty aversion. But there’s something more going on: Power.

There was also a common failure here to think through the meaning of conflicts of interests like Merck’s. Pharmaceutical companies fleecing states and endangering people through spin science is standard corporate corruption fare. Is there a reason to believe that Merck didn’t know it stood to make billions more from novel drugs than from old, generic ones? What’s the source of the strawman conspiracy Griffin puts out here? Maybe someone who has done a historiography of Covid conspiracies could point it out; or maybe I shouldn’t ask that on the Internet.

Speaking of which, let’s talk about a few facets of recent political and scientific theater about possible post-Covid vaccine heart problems in young males.

Example 2: Young male vaccine myocarditis

Part 1: Political Theatre

This example is juicy for politics nerds, because it appears the Kremlin recently killed off Wagner Group warlord and mercurial coup attempter Yevgeny Prigozhin in a plane crash. BBC reported “Social media linked to the Wagner mercenary group say his private plane was shot down by Russian air defences.” In contrast, RT reported the plane crashed after the pilot — who had suffered post-vaccine myocarditis — suffered a heart attack.

I read this when RT reported it. But now, Google won’t show me the result. RT is blocked on German Twitter, with a statement that:

the account has been withheld in Portugal, Finland, Sweden, Ireland, Slovenia, Czech Republic, Poland, Slovakia, Hungary, Italy, Malta, Germany, Greece, Romania, Netherlands, Bulgaria, Austria, Luxembourg, Latvia, United Kingdom, Denmark, Lithuania, Croatia, Estonia, Cyprus, France, Spain, Belgium in response to a legal demand.

And, according to Forbes, Reuters, and my browser history, I’m wrong. The article was a fabricated screenshot that never really existed. So I must be imagining having seen it in my news aggregator and Google search results. Or maybe it came through from those sources in the context of a fabricated social media screenshot, and I missed the source. Maybe it even got through on Google News search results? I remember, by chance, searching for a news hook for another piece just 28 minutes after the plane crash was reported. I thought I came across the RT article then. So if it wasn’t real and I didn’t click through and read it on RT, my memory is wrong…

My memory is wrong. Brains are faulty! My brain is no exception. In this case, further investigation revealed the source of the error was the usual one on social media: A friend sent me a screenshot of the RT article over text. I looked at it and assumed it was from the source it said it was from — making the same sort of transitive trust mistake that many people made with voting and crime misinformation on WhatsApp and Facebook in India and Brazil. I didn’t remember that I didn’t read this article on RT myself; I remembered seeing it on RT, and having gotten it from a trusted source while checking my usual ones. (Lateral reading can solve this problem, when you think to do it. But who laterally reads WhatsApp texts?)

This is why we need meta-science. Science applying the lessons of science, to science. Some people are so worried about artificial intelligence gaining self-awareness; but we have not even managed to create self-aware science as a social institution. Such a self-awareness would entail knowledge of the fact that misinformation discourse is political. So research that seeks to combat misinformation often accidentally reproduces it in one form or another. Just like all other social institutions (and parents) generally correct some of the previous generation’s mistakes while reproducing others. Sure, we all want to break the cycle and get it right. But we’re also all only human. What does this mean for myocarditis and Covid vaccines in young males?

In the case of the Prigozhin assassination, it flipped a probable PR hit for Russia in most of the world. Powerful warlord publicly attempts something like a righteous coup, with apparent popular support, and is threatening enough to the power structure that they make concessions to his demands, try and fail to exile him and take his military power, and finally kill him with military force. This looks weak and desperate — he was one meathead, and you had to have your air force belatedly shoot him out of the sky from a safe distance? This is like execution by firing squad on steroids; no single person was responsible for the killing, so no one could defect and screw it up. Was Prigozhin so widely beloved that the Kremlin couldn’t even trust someone to serve him poisoned tea?

So this looks desperate until, that is, you see the (basically contemporaneous) supposed RT coverage, which flips it into a postmodern PR win for the Kremlin, instead (à la Surkov). Because if you think Russian propaganda is ridiculous, you might see the (apparently not really) RT piece and think “Haha! The Russians killed him and then did disinfo with the assassination! Evil geniuses!” But if you buy Russian propaganda, you might see the (still apparently not really) RT piece and think “Those evil Americans and their ignorant/evil bastard allies! First they’ll blame the Russians for the plane crash! Then they’ll vaccinate us all, the murderers!”

And that’s just level 1 of the spin. Level 2 comes next, when you learn the supposed RT article was a fake. Dominant establishment American propaganda customers who follow it out this far might then think “Haha, the Internet made fun of how crazy the Russians are, blatantly murdering dissidents and putting out disinfo about it.” Dominant establishment Russian propaganda customers might think “Just because you put out disinfo on vaccine side effects, doesn’t mean there aren’t vaccine side effects.” Neither of these reads is necessarily wrong. But they both underscore Russian power.

In both spheres of influence, Russia net gains public opinion manna from this prank where previously it had stood to lose it by looking weak and desperately threatened at its core. This is high-level, artful spin. It was also a logistics feat: That an ostensible RT story immediately went viral while Russia Today was blocked across the Western world following the Russian invasion of Ukraine is rather spectacular. You can’t find out about Russian propaganda on Covid vaccines directly from Russian sources by asking Youtube or Google (not without a VPN); and yet, in the face of widespread, cross-platform censorship, an ostensible RT story went around the world with the news of Prigozhin’s assassination, making the Kremlin look powerful (and indeed perhaps showing its power) when it had just been made to look weak.

Moving on, note that part 1 of this item is political theater, part 2 is scientific theater, and both are about misinformation. We’re used to thinking of misinformation as a political theater game and not as much of thinking of it as a scientific theater game. That it is both, is both a political and a scientific argument. Because applying cognitive science in methodology is both a political and a scientific game.

Part 2: Scientific Theater

In the case of actual myocarditis risk and Covid vaccines in young males, let’s take a look at the myth, the myth-debunking, and some possible debunking of the debunking. In their Oct. 2022 Covid myth-busting piece, Independent SAGE addresses this under “MYTH 6: ‘Vaccines are worse than the disease,’ ” which states:

The side-effect most commonly discussed now is myocarditis which is usually mentioned within the context of the mRNA vaccines. Myocarditis is inflammation of the heart muscles (myocardium) that can cause chest pain, shortness of breath and rapid or irregular heart rhythms (arrhythmias). Here’s some information on the risk of this condition post-vaccination: 

It is reported that for every 1 million vaccine doses administered among males aged 16 to 29 years old, we can expect approximately 5-10 myocarditis events. In younger males aged 12 to 15, the risk is less than 5 per 100,000 vaccines administered. [Bold mine.] The risks in other age groups and females are much lower than this. The “background risk” of myocarditis (i.e., the risk of getting it from other viruses) is ~0.2 to 2.2 per 1 million infections. So there is a link between vaccines and myocarditis in younger males. 

However, it’s also true that the virus itself can cause myocarditis. The risk of myocarditis is 11 fold higher in the 28 days following a SARS-CoV-2 positive test. Other heart problems such as pericarditis and cardiac arrhythmias are also higher following a positive SARS-CoV-2 test. 

Myocarditis, like any disease, is not all or nothing, ranging from no symptoms (asymptomatic) to severe. Thankfully, myocarditis after the vaccine is relatively mild; only 2% of people have to go to intensive care and nearly all people fully recover. There have been deaths where vaccination was implicated, but the causal link (i.e. whether it was caused by the virus or the vaccine) is still under investigation and unclear. 

On the other hand, the severity of myocarditis after a virus infection is much higher; approximately 50% of people go to ICU, 25% do not fully recover, and 11-22% die. 

Citations needed.

Again, I used a shortcut instead of digging up and critiquing all these original sources. The shortcut was knowing Vinay Prasad has been a vocal critic of Covid science, science communication, and policy, including on this front, and thus Googling “prasad vinay myocarditis risk.” Among the top results are this study arguing most myocarditis articles didn’t stratify enough (i.e., on sex, age, dose number and manufacturer — which you’d think would be a minimum set; see “COVID-19 vaccine induced myocarditis in young males: A systematic review,” by Benjamin Knudsen and Vinay Prasad, Eur J Clin Invest. 2023 Apr;53(4):e13947).

Knudsen and Prasad report:

The highest incidence of myocarditis ranged from 8.1-39 cases per 100,000 persons (or doses) in studies using four stratifiers. Six studies reported an incidence greater than 15 cases per 100,000 persons (or doses) in males aged 12-24 after dose 2 of an mRNA-based vaccine.

There’s a big difference between 5-10 post-vaccine myocarditis cases per 100,000 young males (Indie SAGE’s count) and 8-39 (Knudsen and Prasad’s). But it’s still a rare problem, and the overarching question of comparative advantage remains: Is there less myocarditis risk post-vaccine or post-infection for young males?

Another top search result addresses this: “Why a major study on myocarditis risk following COVID vaccination should not influence public-health policy,” an opinion article by Paul S. Bourdon and Spiro P. Pantazatos, Front. Med., 23 March 2023, Sec. Infectious Diseases: Pathogenesis and Therapy. (This journal is on a predatory journals list. But scientific publishing is so corrupt and broken that I’m not sure what that really means.)

Bourdon and Pantazatos focus on the apparent source of the net risk reduction claim (i.e., post-vaccine myocarditis risk in young males is greater than baseline but less than post-infection). That source is Patone et al’s “Risk of Myocarditis After Sequential Doses of COVID-19 Vaccine and SARS-CoV-2 Infection by Age and Sex,” Circulation, 2022 Sep 6;146(10):743-754:

In a population of > 42 million vaccinated individuals, we report several new findings that could influence public health policy on COVID-19 vaccination. First, the risk of myocarditis is substantially higher after SARS-CoV-2 infection in unvaccinated individuals than the increase in risk observed after a first dose of ChAdOx1nCoV-19 vaccine, and a first, second, or booster dose of BNT162b2 vaccine. [Bold mine.]

…[A]lthough we were able to include 2, 230, 058 children age 13 to 17 years in this analysis, the number of myocarditis events was small (56 events in all periods and 16 events in the 1 to 28 days after vaccination) in this subpopulation and precluded a separate evaluation of risk.

Bourdon and Pantazatos correctly interpret this reported data differently than Patone et al:

Thus, it appears there were no positive-test-associated cases of myocarditis among members of their study population in the age range 13–17. This is consistent with data in eTable 7 from a study by Karlstad et al.(7) showing 0 cases of myocarditis associated with SARS-CoV-2 infection for males and females in the age range 12–15. Thus, Patone et al.'s data, together with the data from Karlstad et al.'s study, suggests that for children between 12–17 the risk of myocarditis after vaccination is higher than that after SARS-CoV-2 infection (contrary to Patone et al.'s finding, quoted in our introduction above, suggesting the opposite is true in general).

The outcome (myocarditis) is so rare in this population (young males) that the data from Patone et al alone don’t indicate that there definitely is or is not an association between vaccination and greater or lesser risk. They just show that there may be an increased risk from vaccination. Looking to the other reference Bourdon and Pantazatos cite in this reinterpretation, Karlstad et al, we find:

Among males 16 to 24 years of age, adjusted IRRs were 5.31 (95% CI, 3.68-7.68) for a second dose of BNT162b2 [Comirnaty; BioNech and Pfizer] and 13.83 (95% CI, 8.08-23.68) for a second dose of mRNA-1273 [Moderna], and numbers of excess events were 5.55 (95% CI, 3.70-7.39) events per 100 000 vaccinees after the second dose of BNT162b2 and 18.39 (9.05-27.72) events per 100 000 vaccinees after the second dose of mRNA-1273. Estimates for pericarditis were similar.

… Results of this large cohort study indicated that both first and second doses of mRNA vaccines were associated with increased risk of myocarditis and pericarditis. For individuals receiving 2 doses of the same vaccine, risk of myocarditis was highest among young males (aged 16-24 years) after the second dose. These findings are compatible with between 4 and 7 excess events in 28 days per 100 000 vaccinees after BNT162b2, and between 9 and 28 excess events per 100 000 vaccinees after mRNA-1273. This risk should be balanced against the benefits of protecting against severe COVID-19 disease.

Seeing the Pfizer versus Moderna estimated compatability intervals made me wonder if Novavax still carries a pediatric male myocarditis risk; it does. Johnson & Johnson’s Janssen does, too. There is no free lunch with myocarditis risk in young males post-Covid vaccine. But Moderna looks like it may pose more risk than the alternatives.

Karlstad et al was a large Scandinavian cohort study (Denmark, Finland, Norway, and Sweden). The Scandinavian countries had very different Covid policies (oy, Sweden). So it would be nice to see country-level subanalyses here. That might be a way to leverage different infection exposures.

But if we assume that infection was endemic among schoolchildren generally, it becomes really hard to figure out how we would tell if vaccination reduced post-infection myocarditis rate. We would need something like rolling vaccination access to assess that, maybe. And we need to think about selection bias, too, as vaccinated and unvaccinated people may systematically differ in ways that could causally influence other health outcomes like myocarditis/pericarditis. So there’s a lot it seems we don’t know from a quick glance at some underlying literature here. Like the post-infection myocarditis risk, without vaccination on the table. If only someone had published a paper on pre-vaccine roll-out post-Covid myocarditis!

Searching for “post-covid myocarditis in unvaccinated males” reveals that Tuvali et al did just that, looking at Israeli data from March 2020 to January 2021 (vaccination roll-out was Dec. 20, 2020):

Nine post-COVID-19 patients developed myocarditis (0.0046%), and eleven patients were diagnosed with pericarditis (0.0056%). In the control cohort, 27 patients had myocarditis (0.0046%) and 52 had pericarditis (0.0088%). Age (adjusted hazard ratio [aHR] 0.96, 95% confidence interval [CI]; 0.93 to 1.00) and male sex (aHR 4.42; 95% CI, 1.64 to 11.96) were associated with myocarditis. Male sex (aHR 1.93; 95% CI 1.09 to 3.41) and peripheral vascular disease (aHR 4.20; 95% CI 1.50 to 11.72) were associated with pericarditis. Post COVID-19 infection was not associated with either myocarditis (aHR 1.08; 95% CI 0.45 to 2.56) or pericarditis (aHR 0.53; 95% CI 0.25 to 1.13). We did not observe an increased incidence of neither pericarditis nor myocarditis in adult patients recovering from COVID-19 infection.

Two problems: First, Tuvali et al were only looking at adults. And second, they misinterpreted their findings. Their misinterpretation reflects the common cognitive bias of dichotomania, and its typical expression in researchers writing off a substantial possible effect as null because the 95% compatability interval is wide. This evidence actually suggests Covid infection may have increased myocarditis and pericarditis, especially myocarditis in males — the by-now familiar Covid story. Let’s try again…

Searching for “pediatric post-covid myocarditis males” on Google turns up, among other top hits, this Children’s Hospital of Philadelphia “Get the Facts” webpage which, as far as I can tell, is not based on sufficient evidence, but illustrates the myth-debunking discourse structure and lingo very well. Putting the same search terms into PubMed still doesn’t appear to yield a decisive answer. Ideally, we want a base rate of pediatric and young male post-covid myocarditis and pericarditis in males. This is basic epidemiological data. Hospitals have it. Selection bias is going to be a problem vis-a-vis vaccination. But these data should be there in published form with selection bias problems we can talk and think about; and yet, as far as I can tell, it’s not. Why not?

The pandemic involved a lot of chaos. Data on human beings, by human beings, tends to be messy anyway. Here we would really want to see something like a good pre and post vaccine roll-out/rolling access pediatric male myocarditis and pericarditis analysis including good data on infections — data that no one seems to have. No surprise that the closest contenders seem to be Scandinavian and Israeli researchers (scientists from relatively well-organized states in terms of vaccine response and health datasets). They still don’t tell us what we need to know.

It wouldn’t be impossible to just ask people if they thought they had Covid in this context, instead of asking about testing, and put some stock in what they report. Triangulating self-reports is possible and probably makes sense here. But no one has done that, it seems. Scientists do have a habit of not believing ordinary people when they say what they experience. Sometimes this mistrust is useful; sometimes it is senseless. Here it’s arguably the latter, because a lot of people in a pandemic are going to get sick and not bother confirming what their illness is. Better to tackle that measurement bias with a little data and thought, than to ignore it.

So no answer still, but there are interesting hits, as usual, in PubMed. Here are the highlights:

• Aviel et al published a “Case Series of Myocarditis Following mRNA COVID Vaccine Compared to Pediatric Multisystem Inflammatory Syndrome: Multicenter Retrospective Study” (Vaccines (Basel) 2022 Jul 29;10(8):1207). This seems questionable: We need to know denominators to work out relative risks; instead collecting case reports of different sorts of worst-cases and presenting them for comparison is misleading. Otherwise readers are going to anchor on a direct comparison that’s wrong. A direct comparison like this one:

  • “Patients with post-vaccination myocarditis had a shorter duration of stay in the hospital (mean 4.4 ± 1.9 vs. 8.7 ± 4.7 days) and less myocardial dysfunction (11.1% vs. 61.5%), and all had excellent outcomes as compared to the chronic changes among 9.2% of the patients with PIMS.”

  • Ok, but if post-vaccine myocarditis happens on the order of 30/100,000 times, and severe post-Covid Pediatric Multisystem Inflammatory Syndrome — sometimes noted to resemble or thought to be on a continuum with Kawasaki Disease at the pole, but without male dominance — happens on the order of maybe 14.7/100,000 times, with 40-80% of these patients developing symptomatic myocarditis… Then a rational parent might still rather gamble their child’s heart on Covid than on a booster. And another rational parent might rather gamble the other way, arguing perhaps that Kawasaki is a pole, there are more people on the continuum, and so maybe the risks are equal but the post-vaccine damage is more controlled if there’s less of a massive immune response causing it.

• It’s worth noting that the unlucky few young people who experience post-infection Kawasaki-like disease do seem to recover, including in terms of cardiac abnormalities. At least, that is what the medical literature says. But the medial literature is written by human beings who want to believe that happy ending; in reality, we wouldn’t know yet if some of these kids dropped dead at 30 from heart attacks.

• It’s also worth noting that the unlucky few children and young adults who experience post-vaccine myocarditis probably also fully recover. But there are case reports of “Relapsing myocarditis following initial recovery of post COVID-19 vaccination in two adolescent males” and indications of persistent scarring. Again, unfortunately, we wouldn’t know yet if some of these kids dropped dead at 30 from heart attacks. Much less the relative proportion of young males who had post-vaccine versus post-infection myocarditis who went on to suffer early cardiac death perhaps decades later.

• Arguably, this is what we really care about here: Net medium- to long-term survival differences between males who developed post-vaccine myocarditis, and those who developed post-infection myocarditis, controlling for vaccination. No one has done this comparison. It won’t be possible, probably, for decades.

• We could also be looking for differences between boys and men who develop post-vaccine and/or post-infection myocarditis, and those who don’t, since technically what we care about could be relative risks for a vulnerable subgroup or subgroups, and the net vaccine-infection risk balance could differ for that subset in relation to the whole.

Overall: We should get more and better data comparing cardiovascular and other outcomes post-infection and post-vaccine; but we won’t have the data we really want on this for a long time. Based on its possible heightened risks for no established comparative benefits, it’s not clear to me what the empirical basis is for healthcare professionals offering young males the Moderna vaccine — or science communicators dismissing related concerns as myth. And Indie SAGE’s “myth-busting” about the Covid vaccine-myocarditis risk in young males appears to be insufficiently evidence-based.

At a larger level, my best guess is that the cognitive bias at work here in Indie SAGE’s clear pronouncements where definitive evidence seems lacking is again dichotomania, along with the emotional motive to assert control. We all want to believe we can do something sensible to protect our children, like vaccinating them against disease. It’s easier to believe this works, than to believe the net risks of some vaccinations may outweigh the net risks of disease in some subgroups — but we’re not really sure, and possibly won’t know for decades.

So this looks like just another example of uncertainty making people uncomfortable. People often respond to uncertainty with denial (including attack), rather than trying to resolve or accept it. Then, they claim that science says stuff it doesn’t say in order to tell a preferred story of powerful social networks (here, pharmaceutical companies’ interests in selling vaccines, and broader state-corporate interests in containing the damage of the pandemic to the economy by making it appear as though the risk-benefit ratio of vaccine mandates is settled when it’s not). Again, what Indie SAGE has done here is not good science communication. It is spin.

As usual in science and other social enterprises, a large part of the spin comes from the focus, aka setting the terms of the discourse. Indie SAGE could have synthetic-directive work with a critical-reflective component. That might have featured, for instance, up-to-date comparisons of possible post-vaccine heart risks for boys and young men who got different vaccines. There are lots of open methodological questions in this terrain. Instead, they dismissed the possibility of preventable harm. There is a chance that some vaccines are considerably safer than others for some people, which highlights the troubling and still-live possibility that some cases of Covid may also be safer for some people than some vaccines. Denying the reality of that uncertainty is establishment propaganda.

It also hinders our ability to do more, better preventive medicine. Heart problems in boys and young men are a (rare) problem after both Covid and the vaccine; the debate is over the net risk-benefit ratio, the scale and significance of the problem, and what denominators and covariates to do what with. Misinformation discourse denying that debate exists do a disservice to these young people, and their families and friends. Because there are next places to run with this ball.

NSAIDs have been studied in this context. The recent human subjects data show they might hurt and might help, but they skew toward hurting; and that’s consistent with earlier animal evidence showing deleterious effects. So you could say more research is needed, but maybe we want to look somewhere else.

Some other logical interventions to explore here include established prophylaxis like red algae or lysine to prevent possible post-infection problems. The point is not that I have the right answer and it’s lysine. It is, rather, that arguing for more/less vaccination in these subgroups is one plane of debate, and we could be asking other questions about (for instance) measuring selection biases and preventing viral replication instead.

It’s defensible to ask different questions than these, too. What’s not is doing nothing and denying the problem in all its dimensions of uncertainty, because it’s inconvenient that previously healthy boys and young men might drop dead at 30 from vaccines that corporates and states want to mandate entire populations take annually with limited evidence. That would seem to undervalue the affected people’s well-being for political purposes.

In addition, it will cause a helluva anti-vaxx backlash if it comes out in a few decades that post-vaccine myocarditis did indeed cause permanent damage leading to net early deaths — while the establishment responded with denial to suggestive evidence that this might be so. Acknowledging the uncertainty opens the door not just to building better knowledge about how to mitigate possible net risks, but also nurturing public trust in scientists and our communication about our work. We have to get better at saying we don’t know, what we don’t know. Because it’s the truth, and sometimes the truth comes out.

All this has hopefully obvious implications for mandatory vaccine boosters (item 3). Mandating boosters with an uncertain risk-benefit ratio for boys and young men seems like insufficiently evidence-based public policy that may do preventable harm including hurting boys and young men as a group — as well as harming public trust in institutions down the line. If the risks of Covid to young people are pretty low, the risks of vaccination to young males may be non-negligible, and the evidence about the net risks is ambiguous, uncertain, exegetical, incomplete, and in flux — then why is it someone else’s prerogative to tell parents and young men what they have to do with their bodies? What would this discourse look like if the genders were flipped, and we were having a conversation instead about female bodily autonomy and state power?

Speaking of which, what about girls and women?

Example 3 (was originally item 6): Reproductive effects

A lot of girls and women noticed that Covid vaccines changed our periods. Cycle abnormalities like unexpected bleeding were common. The response of a lot of healthcare practitioners and science communicators was essentially: “It’s all in your head, silly female. Now take your medicine and go away; we have real work to do.”

Kate Clancy, University of Illinois Professor of Anthropology and author of the imminently readable and fascinating Period: The Real Story of Menstruation (WaPo adaptation), wasn’t having it. As related in the book, she and her collaborators conducted online survey research (1, 2) finding “about 40% of cisgender, currently menstruating respondents experienced heavier menstrual bleeding after getting the vaccine… A majority… on long-acting reversible contraception had breakthrough bleeding, as well as almost 40 percent of those on gender-affirming hormones and two-thirds of postmenopausal people” (p. 129).

The point is not that this online survey data should be expected to generalize. It may well be that people who had post-vaccine period problems were over-sampled due to selection bias. But even with a 15% selection bias skewing toward people who experienced this sort of problem, we’re still talking about an effect on 25% of half of the population. This suggests that post-vaccine reproductive effects aren’t an established myth to be busted. They seem to be a common female experience.

Indie SAGE, like most consensus science communicators, deny this experience. In their “myth-busting,” they set up the reproductive effect issue through a narrower frame: “MYTH 7: ‘Vaccines affect your fertility.’ ” Then they argue data suggest vaccination did not reduce chances of conception or affect ovarian reserve, and remind readers:

Whilst the vaccine doesn’t reduce fertility, infection with SARS-CoV-2 is associated with a short-term reduction in fertility in males. Importantly, catching the virus when pregnant increases the chance of going to intensive care, having an early birth and that the baby will also go to intensive care, so it’s very important for pregnant women to be vaccinated. 

As far as I know, these statements are well-supported by existing evidence. These facts seem important for people with reproductive health concerns to know. It’s great that Indie SAGE passes them on.

The problem is that everything Indie SAGE says in “busting” the “myth” here, while probably right on its own terms, spins evidence to emphasize the preferred causal narrative that the vaccine has no reproductive effects. Millions of girls and women experienced otherwise. It is an abuse of power to ignore that reality.

Maybe Indie SAGE didn’t know. There’s nothing about periods or menstruation on their website, as far as I can tell. Their gender balance doesn’t appear super skewed. Maybe the female committee members were quiet about post-vaccine period problems they and their female friends may have had, or maybe they didn’t have them or hear about them. Maybe they don’t think heavy menstrual bleeding counts as a reproductive problem, if it doesn’t seem to change the number of live births downstream.

The fact remains that vaccines can have what most people would call reproductive effects including unexpected menstrual changes that may range from inconvenient to painful or distressing, most notably heavy bleeding — and Covid vaccines appear to have done that. Again, the point is not that this science is right and that science is wrong. In this case, they both seem to be right.

The point, rather, is that the terms, focus, and interpretation of evidence are functioning as spin in a political game. Just because the theater is scientific, doesn’t make us into particles or gods who magically become neutral when we look at the world. And just because we call something science communication, doesn’t make it scientific. Science requires critical thinking. Parroting a party line doesn’t fit the bill. Doing so at the cost of gaslighting half the population is uncool. It also probably stands to hurt your credibility with that half.

At the same time, time and resources are finite. On all these points, Indie SAGE just did some research and stopped somewhere. We are all stopping somewhere when we verify facts, all the time. There is no science without trust. Choosing who and what to trust is political. This is scary. It should make us nervously look for the exits — from bias, from error, from spin of all kinds. There are none.

Recap

Misinformation discourse often falls short of the logical and scientific standards to which it (implicitly or explicitly) claims to hold others. This ironic double standard could be remedied by applying cognitive science to science, as leading statistics reformers suggest. Yet, there is no exit from our fallibility and non-neutrality as human beings. This makes doing science dangerous. We want to get it right, but we’re always ever only getting it less wrong (at best).

So we have to talk more about mistakes, ambiguity, and uncertainty. Even (or perhaps especially) when what we want to do is reassure people that we’re sure what the best course of action is — because we want them to do what we want. This is an authoritarian impulse, even when it’s well-intentioned. It has no place in science, or in science communication.

Except, apparently, it does. The problems in Indie SAGE’s myth-busting are common problems in misinformation discourse. They show the seams of power in science. We can do better, but only to a point.

As Yates writes, “Sadly, it is often the case that even when we think we are reasoning logically, our intuition lets us down… We all think we are so good at being logical that we rarely stop to step back and question our own reasoning” (p. 358).

Postscript

It is, in a way, fitting that Vinay Prasad has just published the post “Do not report COVID cases to schools & do not test yourself if you feel ill.” Suffice it to say that this is not a synthetic-directive work with particularly critical-reflective components. It makes me think of anecdata from my circles in which mothers lost pregnancies — some early, some quite late — after Covid infections passed through their schoolage kids and their households. These losses changed these women’s lives and families, and didn’t get counted as deaths. They were mourned.

What Prasad proposes here puts particularly vulnerable groups, including immunocompromised and pregnant people, at risk of serious harm. It gives pregnant mothers of schoolage kids no way to protect themselves or their unborn children. Yet, the great pain and shame associated with pregnancy loss for many may keep this part of this story largely publicly untold for some time. I wish these women had the power of voice. But pain and shame make silence. All I can do is point out where power is at work in the discourse, as it always is.

Prasad’s latest remains a shot in what he sees as the dark of the authoritarianism of the Covid science-policy discourse. That authoritarianism has a real empirical basis in the spin science of the Covid misinformation discourse critiqued here. Both things are true: On one hand, Covid threatens the well-being of some groups more than others, with often tragic consequences. And, on the other hand, Covid discourse (like most science communication) typically presents spin as certain truth, with scientists often promoting a preferred narrative that is not established as fact — and policy-makers often enacting policy based on this narrative. Bias and power shape information. It’s easier to lapse into synthesizing what one sees as the facts into one’s own, corrected directive, than it is to pull back to the plane of critical reflection on how limited we all remain.

Sound familiar? Arete, hubris, ate, nemesis – virtue, arrogance, fatal mistake, divine punishment. The recurrent structure of world literature classics, failed policies, and reactionary political movements alike.