Face Masks: Much More Than You Wanted To Know

There’s been recent controversy about the use of face masks for protection against coronavirus. Mainstream sources, including the CDC and most of the media say masks are likely useless and not recommended. They’ve recently been challenged, for example by Professor Zeynep Tufekci in the New York Times and by Jim and Elizabeth on Less Wrong. There was also some debate in the comment section here last week, so I promised I’d look into it in more depth.

As far as I can tell, both sides agree on some points.

They agree that N95 respirators, when properly used by trained professionals, help prevent the wearer from getting infected.

They agree that surgical masks help prevent sick people from infecting others. Since many sick people don’t know they are sick, in an ideal world with unlimited mask supplies everyone would wear surgical masks just to prevent themselves from spreading disease.

They also agree that there’s currently a shortage of both surgical masks and respirators, so for altruistic reasons people should avoid hoarding them and give healthcare workers first dibs.

But they disagree on whether surgical masks alone help prevent the wearer from becoming infected, which will be the focus of the rest of this piece.

1. What are the theoretical reasons why surgical masks might or might not work?

Epidemiologists used to sort disease transmission into three categories: contact, droplet, and airborne. Contact means you only get a disease by touching a victim. This could be literally touching them, or a euphemism for very explicit contact like kissing or sex. Droplet means you get a disease when a victim expels disease-laden particles into your face, usually through coughing, sneezing, or talking. Airborne means you get a disease because it floats in the air and you breathe it in. Transmission via “fomites”, objects like doorknobs and tables that a victim has touched and left their germs on, is a bonus transmission route that can accompany any of these other methods.

More recently, scientists have realized that droplet and airborne transmission exist along more of a spectrum. Droplets can stay in the air for more or less time, and spread through more or less volume of space before settling on the ground. The term for this new droplet-airborne spectrum idea is “aerosol transmission”. Diseases with aerosol transmission may be spread primarily through droplets, but can get inhaled along with the air too. This concept is controversial, with different authorities having different opinions over which viruses can be aerosolized. It looks like most people now believe aerosol transmission is real and applicable to conditions like influenza, SARS, and coronavirus.

Surgical masks are loose pieces of fabric placed in front of the mouth and nose. They offer very good protection against outgoing droplets (e.g. if you sneeze, you won’t infect other people), and offer some protection against incoming droplets (e.g. if someone else sneezes, it doesn’t go straight into your nose). They’re not airtight, so they offer no protection against airborne disease or the airborne component of aerosol diseases.

Respirators are tight pieces of fabric that form a seal around your mouth and nose. They have various “ratings”; N95 is the most common, and I’ll be using “N95 respirator” and “respirator” interchangeably through most of this post even though that’s not quite correct. When used correctly, they theoretically offer protection against incoming and outgoing droplet and airborne diseases; since aerosol diseases are a combination of these, they offer generalized protection against those too. Hospitals hate the new “aerosol transmission” idea, because it means they probably have to switch from easy/cheap/comfortable surgical masks to hard/expensive/uncomfortable respirators for a lot more diseases.

Theory alone tells us surgical masks should not provide complete protection. Coronavirus has aerosol transmission, so it is partly airborne. Since surgical masks cannot prevent inhalation of airborne particles, they shouldn’t offer 100% safety against coronavirus. But theory doesn’t tell us whether they might not offer 99% safety against coronavirus, and that would still be pretty good.

2. Are people who wear surgical masks less likely to get infected during epidemics?

It’s unethical to randomize people to wear vs. not-wear masks during a pandemic, so nobody has done this. Instead we have case-control studies. After the pandemic is over, scientists look at the health care workers who did vs. didn’t get infected, and see whether the infected people were less likely to wear masks. If so, that suggests maybe the masks helped.

This is an especially bad study design, for two reasons. First, it usually suffers recall bias – if someone wore a mask inconsistently, then they’re more likely to summarize this as “didn’t wear masks” if they got infected, and more likely to summarize it as “did wear masks” if they stayed safe. Second, probably some nurses are responsible and do everything right, and other nurses are irresponsible and do everything wrong, and that means that if anything at all helps (eg washing your hands), then it will look like masks working, since the nurses who washed their hands are more likely to have worn masks. Still, these studies are the best we can do.

Gralton & McLaws, 2010 reviews several studies of this type, mostly from the SARS epidemic of the early 2000s. A few are underpowered and find that neither surgical masks nor respirators prevent infection (probably not true). A few others show respirators prevent infection, but do not investigate surgical masks (probably right, but useless for our purposes). Two seem relevant to the question of whether surgical masks work:

Rapid awareness and transmission of SARS in Hanoi French Hospital, Vietnam was conducted in a poor hospital that only had surgical masks, not respirators. In the latter stages of the epidemic, 4 workers got sick and 26 stayed healthy. It found that 3 of the 4 sick workers hadn’t been wearing masks, but only 1 of the 26 healthy workers hadn’t. This is a pretty dramatic result – subject to the above confounders, of course.

Effectiveness of precautions against droplets and contact in prevention of nosocomial transmission of SARS is larger and more prestigious, and looked at a cluster of five hospitals. Staff in these hospitals used a variety of mask types, including jury-rigged paper masks that no serious authority expects to work, surgical masks, and N95 respirators. It found that 7% of paper-mask-wearers got infected, compared to 0% of surgical-mask and respirator wearers. This seems to suggest that surgical masks are pretty good.

The meta-analysis itself avoided drawing any conclusions at all, and would not even admit that N95 respirators worked. It just said that more research was needed. Still, the two studies at least give us a little bit of evidence in surgical masks’ favor.

How concerned should we be that these studies looked at health care workers specifically? On the one hand, health care workers are ordinary humans, so what works for them should work for anyone else. On the other, health care workers may have more practice using these masks, or may face different kinds of situations than other people. Unlike respirators, surgical masks don’t seem particularly hard to use, so I’m not sure health care workers’ training really gives them an advantage here. Overall I think this provides some evidence that surgical masks are helpful.

I was able to find one study like this outside of the health care setting. Some people with swine flu traveled on a plane from New York to China, and many fellow passengers got infected. Some researchers looked at whether passengers who wore masks throughout the flight stayed healthier. The answer was very much yes. They were able to track down 9 people who got sick on the flight and 32 who didn’t. 0% of the sick passengers wore masks, compared to 47% of the healthy passengers. Another way to look at that is that 0% of mask-wearers got sick, but 35% of non-wearers did. This was a significant difference, and of obvious applicability to the current question.

3. Do surgical masks underperform respirators in randomized trials?

Usually it would be unethical to randomize health care workers to no protection, so several studies randomize them to face masks vs. respirators. But a few others were done in foreign hospitals where lack of protection was the norm, and these studies did include a no-protection control group.

MacIntyre & Chugtai 2015, Facemasks For The Prevention Of Infection In Healthcare And Community Settings, reviews four of these. Two of the four are unable to find any benefit of either masks or respirators. The third finds a benefit of respirators, but only if nobody tested the respirators to see if they fit, which doesn’t make sense and suggests it’s probably an artifact. The fourth finds a benefit of respirators, but not masks. It seems unlikely that respirators don’t help, so this suggests all these studies were underpowered.

In other words, respirators are better than masks are better than nothing. It would be wrong to genuinely conclude this, because it’s not statistically significant. But it would also be wrong to conclude the studies show masks don’t work, because they mostly show respirators don’t work, and we (hopefully) know they do.

Overall these studies don’t seem very helpful and I’m reluctant to conclude anything from them. In section 6, I’ll talk more about why studies may not have shown any advantage for respirators.

4. Do surgical masks prevent ordinary people from getting infected outside the healthcare setting?

The same review lists nine randomized trials with a different design: when the doctor diagnoses you with flu, she either asks everyone in your family to wear masks (experimental group), or doesn’t do that (control group), and then checks how many family members in each group got the flu.

How did these go? That depends whether you use intention-to-treat or per-protocol analysis. Intention-to-treat means that you just compare number of infections in the assigned-to-wear-masks group vs. the control group. Per-protocol means that you only count someone in the study if they actually followed directions. So if someone in the assigned-to-wear mask group didn’t wear their mask, you remove them from the study; if someone in the control group went rogue and did wear a mask, you remove them too.

Both of these methods have their pros and cons. Per protocol is good because if you’re trying to determine the effect of wearing a mask, you would really prefer to only be looking at subjects who actually wore a mask. But it has a problem: adherence to protocol is nonrandom. The people who follow your instructions diligently are selected for being diligent people. Maybe they also diligently wash their hands, and diligently practice social distancing. So once you go per protocol, you’re no longer a perfect randomized controlled trial. Only intention-to-treat analyses carry the full weight of a gold standard RCT.

According to intention-to-treat, the studies unanimously found masks to be useless. But there were a lot of signs that intention-to-treat wasn’t the right choice here. Only about a fifth of people who were asked to wear masks did so with any level of consistency. The rest wore the mask for a few hours and then get bored and took it off. Honestly, it’s hard to blame them; these studies asked a lot from families. If a husband has flu, and sleeps in the same bed as his wife, are they both wearing masks all night?

Of the three studies that added per-protocol analyses, all three found masks to be useful (1, 2, 3) . Does this prove masks work? Not 100%; per-protocol analyses are inherently confounded. But it sure is suggestive.

The review author summarizes:

The routine use of facemasks is not recommended by WHO, the CDC, or the ECDC in the community setting. However, the use of facemasks is recommended in crowded settings (such as public transport) and for those at high risk (older people, pregnant women, and those with a medical condition) during an outbreak or pandemic. A modelling study suggests that the use of face-masks in the community may help delay and contain a pandemic, although efficacy estimates were not based on RCT data. Community masks were protective during the SARS outbreaks, and about 76% of the population used a facemask in Hong Kong.

There is evidence that masks have efficacy in the community setting, subject to compliance [13] and early use [12, 18, 19]. It has been shown that compliance in the household setting decreases with each day of mask use, however, which makes long term use over weeks or months a challenge […]

Community RCTs suggest that facemasks provide protection against infection in various community settings, subject to compliance and early use. For health-care workers, the evidence suggests that respirators offer superior protection to facemasks.

Parts of this summary are infuriating. If the big organizations recommend that especially vulnerable groups wear masks, aren’t they admitting masks work? But if they’re admitting masks work, why don’t they recommend them for ordinary people?

It looks like they’re saying masks work a little, they’re too annoying for it to be worth it for normal people, but they might be worth it for the especially vulnerable. But then why don’t they just say masks work, and let each person decide how much annoyance is worthwhile? I’m not sure. But it looks like the author basically ends up in favor of community use of surgical masks in a pandemic, mostly on the basis of per-protocol analyses of community RCTs.

5. How do surgical masks and respirators compare in hokey lab studies?

Our source here is Smith et al 2016, Effectiveness Of N95 Respirators Versus Surgical Masks In Protecting Health Care Workers From Acute Respiratory Infection: A Systematic Review And Meta-Analysis. They review some of the same studies we looked at earlier, but then investigate 23 “surrogate exposure studies”, ie throwing virus-shaped particles at different masks in a lab and seeing if they got through. You can find the results of each in their appendix. Typically, about 1 – 5% of particles make it through the respirator, and 10 – 50% make it through the surgical mask. They summarize this as:

In general, compared with surgical masks, N95 respirators showed less filter penetration, less face-seal leakage and less total inward leakage under the laboratory experimental conditions described.

I think in general the fewer virus particles get through your mask, the better, so I think this endorses surgical masks as better than nothing, since their failure rate was less than 100%.

Booth et al, 2013 examines surgical masks themselves more closely. They hook a surgical mask up to “a breathing simulator” and then squirt real influenza virus at it, finding that:

Live influenza virus was measurable from the air behind all surgical masks tested. The data indicate that a surgical mask will reduce exposure to aerosolised infectious influenza virus; reductions ranged from 1.1- to 55-fold (average 6-fold), depending on the design of the mask…the results demonstrated limitations of surgical masks in this context, although they are to some extent protective.

The paper doesn’t discuss how particle number maps to infection risk. Does letting a single influenza virus through mean you will get infected? If so, any reduction short of 100% is useless. I have a vague sense that this isn’t true; your immune system can fight off most viruses, and the fewer you get, the better the chance it will win. Also, even respirators don’t claim to reduce particle load by more than 99% or so, and those work, so it can’t be that literally a single virus will get you. Overall I think modest reductions in particle number are still pretty good, but I don’t have a study that proves it.

6. Is it true that the public won’t be able to use N95 respirators correctly?

Yes.

I remember my respirator training, the last time I worked in a hospital. They gave the standard two minute explanation, made you put the respirator on, and then made you go underneath a hood where they squirted some aerosolized sugar solution. If you could smell the sugar, your respirator was leaky and you failed. I tried so hard and I failed so many times. It was embarrassing and I hated it.

I’m naturally clumsy and always bad at that kind of thing. Some people were able to listen to the two minute explanation and then pass right away. Those kinds of people could probably also listen to a two minute YouTube explanation and be fine. So I don’t want to claim it’s impossible or requires lots of specialized background knowledge. It’s just a slightly difficult physical skill you have to get right.

Bunyan et al, 2013, Respiratory And Facial Protection: A Critical Review Of Recent Literature, discusses this in more depth. They review some of the same studies we reviewed earlier, showing no benefit of N95 respirators over surgical masks for health care workers in most situations. This doesn’t make much theoretical sense – the respirators should win hands down.

The most likely explanation is: doctors aren’t much better at using respirators than anyone else. In a California study of tuberculosis precautions, 65% of health care workers used their respirators incorrectly. That’s little better than the general public, who have a 76% failure rate. Bunyan et al note:

The fitting of N95 respirators has been the subject of many publications. The effective functioning of N95 respirators requires a seal between the mask and the face of the wearer. Variation in face size and shape and different respirator designs mean that a proper fit is only possible in a minority of health care workers for any particular mask. Winter et al. reported that, for any one of three widely used respirators, a satisfactory fit could be achieved by fewer than half of the healthcare workers tested, and for 28% of the participants none of the masks gave a satisfactory fit.

Fit-testing is a laborious task, taking around 30 min to do properly, and comprises qualitative fit-testing (testing whether the respirator-wearing healthcare worker can taste an intensely bitter or sweet substance sprayed into the ambient air around the outside of the mask) or quantitative fit testing (measuring the ratio of particles in the air inside and outside the breathing zone when wearing the respirator). Attempts have been made to circumvent the requirement for fit testing, and it has been suggested that self-testing for a seal by the respirator wearer (see http://youtu.be/pGXiUyAoEd8a for a video demonstration) is a sufficient substitute for fit-testing. However, self-checking for a seal has been demonstrated to be a highly unreliable technique in two separate studies so that full fit-testing remains a necessary preliminary requirement before respirators can be used in the healthcare setting.

Operationally, this presents significant challenges to organizations with many healthcare workers who require fit-testing. Chakladar et al. pointed out that, in addition to the routine need for repeat testing over time to ensure that changes in weight or facial hair have not compromised a good fit, movements of healthcare workers between organizations using different makes of respirators would necessitate additional repeat fit-testing. Fit-testing is likely to remain problematic to health-care organizations for the foreseeable future. In addition to the requirement for fit-testing, ‘fit-checking’ is also required each time the respirator is donned to ensure there are no air leaks.

Is a poorly-fitting N95 respirator better than nothing? The reviewed studies suggest that at that point it’s just a very fancy and expensive surgical mask.

7. Were the CDC recommendations intentionally deceptive?

No, and I owe them an apology here.

I think the evidence above suggests masks can be helpful. Masked health care workers were less likely to catch disease than unmasked ones. Masked travelers on planes were less likely to catch disease than unmasked ones. In per protocol analysis, masked family members are less likely to catch disease from an index patient than unmasked ones. Laboratory studies confirm that masks block most particles. All of this accords with a commonsense understanding of droplet and aerosol transmission of disease.

None of these, except maybe the plane study, tell us exactly what we want to know. The SARS studies were all done in a health care setting, so they don’t prove that regular people can benefit from masks. But health care workers are closely related to homo sapiens and ought to have similar anatomy and physiology. Surgical masks aren’t as complicated as respirators and we can assume most people get them right. And although health care workers are in unusually high-risk situations, that should just affect the magnitude of the benefit, not the sign; obviously the level of risk ordinary people encounter is sometimes relevant, considering they do often catch pandemic diseases. So our default assumption should be that these studies carry over, not that they don’t.

Likewise, most of the community studies were done on family members. Most guidelines already say to mask up if you have a sick family member, so talking about subways and crowds requires a little bit of extrapolation. But again, being in a family is just one form of close contact. It would take bizarre convolutions to even imagine a theory where you can catch diseases from your family members but not from people you sit next to on a train. Our default assumption should be in favor of these results generalizing, not against them.

But the CDC has recommended against mask use. I hypothesized that the CDC was intentionally lying to us, trying to trick us into not buying masks so there would be enough for health care workers.

But that can’t be true, because the CDC and other experts came up with their no-masks policy years ago, long before there was any supply shortage.