On Randomized Trials and Medicine

The thorny, messy topic of why we don't have one for masks as source-control

The pandemic has been ravaging the world for months and months. Where is our randomized trial showing the benefit of masks? There is none. It’s something mask-skeptics often observe. They also point to a randomized study of cloth masks (the only one that exists) that warned against wearing masks, a Senator Rand Paul did recently in a tweet. What’s going on?

I have an article in the Sunday Review in today’s New York Times discussing this. Spoiler: that randomized study doesn’t show that cloth masks are worse than not wearing masks. More broadly, a randomized trial is a logistical hurdle that is practically impossible to overcome at this point, and not just because of ethical reasons. The demand for a randomized trial proving the benefits to mask-wearers rests on one of the most important but least understood facts about why we started recommending masks in the first place: to prevent disease transmission to others. Mask-wearing is not an individual benefit, it’s a community benefit. Further, this discussion reveals some of the underlying reasons for our feeble response to this pandemic: reasons that go beyond the obvious and many failures of the OUTGOING (!) administration.

To have a proper study for masks for source-control, we’d need to enroll communities and do a cluster randomized study—comparing communities, not just individuals. That is both difficult and also with much less explanatory power than one would hope since pathogen is also overdispersed: some people get hit badly by the disease just by chance. That makes causal inference harder.

Here’s a quote from the article: 

Early in the pandemic, medical authorities recommended that only sick people wear masks, so that they wouldn’t infect others. Then it became clearer that people who were not yet showing symptoms, or who had mild symptoms, were also infecting others. That was one key reason universal mask-wearing became recommended, to prevent transmission to others — what’s known as source control. Preventing the mask-wearer from becoming infected, to the degree that it happens, is a secondary benefit.

Randomized trials look at the benefits of the intervention only for the person who is enrolled in the trial. It’s not possible for them to conclude how effectively masks prevent community spread to others — people who are not enrolled in the trial.

To use a randomized trial to measure the effect on the spread of the disease, researchers would have to find a community, divide it randomly into two and cut off all interaction between the two groups. If the communities of mask-wearers interacted with non-mask-wearers, the non-mask-wearers could infect people who would otherwise be protected. The results would be uninterpretable.

I want to say three things here about this. 

First, it is a pity we don’t have solid data from randomized trials. There is an enormous value from randomized trials. We should have had more randomized studies on such non-pharmaceutical interventions from before the pandemic. Clinical trials for non-pharmaceutical interventions in health are relatively rare because there is little to no money to be made from recommending them. Conversely, drugs which require trials to go on the market are  sponsored heavily by pharmaceutical companies. 

Meanwhile, things we know are excellent for many important health and well-being considerations—exercise for combating depression; mask-wearing for reducing respiratory diseases; physical therapy for improving many musculo-skelatal conditions—are overlooked in trials and clinical practice. Meanwhile, drug companies will go out of their way to do and redo clinical trials for expensive drugs with suspect end-points, which are expectations for the drug’s benefits--as long as companies  find any benefit, however small, they can get approval for it. Companies often conduct tests until they get some result which  gives them an excuse to put the drug on the market. Then they will start heavily promoting the drug, often wasting our health-care money for very little benefit.

Second: randomization, a most powerful tool for clinical practice, is so drastically under-utilized during this pandemic that it’s malpractice. The most effective cheap drug we have is dexamethasone—a corticosteroid—that many medical researchers had perfectly solid reasons to think would not work. We have this amazing therapeutic—available at very little cost, and yet able to cut death rates by an astonishing third—only because the United Kingdom’s NHS and leading scientists launched randomization early on for a project called RECOVERY. Our clinical management of COVID-19 has gotten much better now. But in earlier months, the situation was the kitchen-sink approach: desperate doctors tried everything at hand, in various combinations. If we had more proper randomization and tracing of these various approaches, we’d have more dexamethasones at hand. The WHO is leading some randomized trials at the moment, but we have little to none going on in the United States.As we learn more about some effective clinical practices, it’s getting harder to do novel ones: ethically, we cannot withhold known best practices from patients. It’s possible that we have found ourselves stuck at a local optimum, but much less than the upside potential we might have had, had we tried proper randomization from the beginning, when we had little to no idea what worked anyway, we could have conducted randomized trials. We should have.. This oversight will go down as yet another major failure of our health infrastructure and response to this pandemic.

Third, because of the above two problems, many critics sometimes spend time and energy lamenting the lack of randomized trials. While randomized trials would be nice to have, we don’t need them as badly for masks for source-control because we have something else that we often lack for drugs for which we absolutely must have randomized clinical trials. For masks, we understand mechanisms and the underlying physics of aerosol/droplet spread. 

The effectiveness of some interventions can be shown even without randomized clinical trials. How? First, there must be a physical theory of the mechanism of the intervention. In this case, the theory holds that the disease is spread by virus-laden droplets and aerosols--and that masks dampen the spread of droplets and aerosols that we emit out of our mouths when we breathe, talk, shout, and sing. Second, we must have a way to test and examine that mechanism: in this case, it’s through lab studies, which we definitely do have, finally, at this stage of the pandemic. Third, there should, hopefully, be indirect epidemiological evidence. Do mask-wearing places fare better than non-mask wearing places? As I write in the piece, this issue is thorny because of confounding effects: places where most people  wear masks are probably doing many other things as well. 

This recent epidemiological study out of Kansas (which did not make it into my piece because it just came out) is encouraging. Counties with a mask-mandate that were seeing increasing cases before a mask mandate (and thus likely were not taking other behavioral interventions like distancing) saw a dramatic decrease in cases starting about 14 days after the mask mandate was implemented. That was compared with places that had seen an increase but had gotten it under control without a mask mandate—presumably with people being scared straight because of their increasing cases—saw their cases surge again.

There are many epidemiological studies like this showing a benefit from masks.  While the aforementioned confounding presents a problem, these studies do help our available evidence for recommending masks . That’s true especially since, to be able to recommend an intervention without a randomized clinical study, we should be reasonably sure that the intervention does not have a significant downside. The rule ‘Do no harm’ is at the root of medical ethics for good reason: it’s too easy to do harm when giving people stuff to ingest, and when cutting into people. That’s particularly the case  because, too often, we don’t really have a clear molecular understanding of all the complex things that can go wrong with a disease. Randomized clinical trials give us a good way to try to get at this messy but important complexity. But doctors are, understandably, trained to prioritize the rule to  “do no harm” in their ethical reasoning. 

But here’s the problem with that mentality: it can lead to looking for harms and finding them even when no major ones plausibly exist.  We are seeing that training combined with an unfortunate strain of cynicism that seems to have a lot of influence over both public health and medicine: a mistrust of people. There is a widespread propensity for pop-psychology over an actual understanding of human behavior. Those are the conditions under which all sorts of alleged “harms” from masks spread among the medical community. Many doctors claimed wearing masks would lead people to be more reckless (nonsense). Many said masks,  would be of no benefit if people did not wear them correctly (that claim made no sense). We heard that people would self-contaminate (how? The descriptions of this would happen were sometimes ridiculous).Finally, many doctors said that if we recommended masks, people would hoard them rather than be responsible with them, and wear cloth masks while we solved the PPE problem for healthcare workers.  

That was the basis of my very first piece on masks, the one that appeared in March: the mistrust and alleged harms of masks that muddied the messaging so terribly that we are still paying a huge price for it. I also have a longer section on the sociology of the question in this preprint (in the final stages of peer-review)—look for the section titled “Sociological considerations”. 

My piece in Sunday Review uses a misleading tweet by Senator Rand Paul to explain why the often-cited study allegedly showing harms from cloth masks does nothing of the sort (to understand how we should respond to PPE shortages, the study compared barely-washed cloth masks to surgical masks for protecting medical workers and, unsurprisingly, found disposable surgical masks were better). There is a tendency to think that this kind of erroneous thinking happens only “on that side,” that it’s a function of the ideology that belongs to people who supported the OUTGOING (!) president. In reality, while opposing masks has now become an ideological component of pandemic-denialism, some the problems I outline above permeate not just supporters of this president, but much of Western medical establishment as well. This is also why the instruction to “just follow the science” isn’t enough to address this pandemic. Yes, we should absolutely follow the science, but here’s the awful truth: we do not have a “science” that is fully up to the challenge, especially when it comes to understanding the intersection between human behavior and the pandemic, and the many complications and twists of the failings of our expert communities and how they relate to society. That task remains ahead of us.