This was a helpful example for showing how understanding methods and carefully reading the actual study are so important and can prevent misinterpretation of results. Venting just a little here, but it has been very frustrating discussing post-vaccination behavior with roommates who are heavily engaged with a lot of news sources that have been heavily emphasizing the potential threat of variants. It has been very difficult to acknowledge that there are real concerns there, but that our vaccinations (particularly the mRNA) have shown little reason to panic to this point or fret that the pandemic will never end. And also there seems to be a poor understanding as you've been saying that a small number of breakthrough cases aren't necessarily cause for alarm either.
I'm not offering analysis here, just expressing some exasperation... and also saying thank you!
I share your exasperation! Even as I write those words, I'm reminded that I shouldn't be surprised as every "corner" we turn through this process has been quite difficult - any time we're adjusting our behavior, the social dynamics of navigating the change can be so difficult - especially if you are someone who is sort of in-between the extremes and wanting to make decisions guided at least partially by science and data.
I'm part of a very cautious homeschool pod and it is so hard to discuss potential (slight!) behavior changes for our last two months as some people want to maintain masking between fully vaccinated (and cautious) adults outdoors. As someone is interested in both the science (doesn't get much more low risk than fully vaccinated, cautious, adults outdoors) and social dynamics (some degree of autonomy is needed for any successful group to stay healthy) - this seems like such an easy change to make and yet fear over breakthrough cases and variants and "the data is so new" and you never know.... make for a more cautious end of the year even as we are mostly fully vaccinated adults in our group. Sigh.
Yes, exactly! I feel the same way. It's hard (both as a matter of reaching the right understanding and of carrying that understanding into social situations) to try to adhere to the science, rather than one of the oppositional extreme interpretations of it. Thanks, Katie!
I think we need a medical/scientific equivalent of "snopes" to cover scientific papers, which journalists would consult before printing their stories. Over and over again it's clear that journalists are not up to the task of understanding these papers. Perhaps they could come together to fund such an organization which they could all make use of.
Given that these news misreports are the issue with the Covid coverage, which is a much smaller set of papers and is the set we are concerned about here *and* is the purview of Snopes itself (news misreporting), I propose that Zeynep approach Snopes with this proposal and offer to recommend some Snopers to do these reviews. Plus the Snopers could submit perhaps shorter versions of the ones Zeynep does herself.
It seems like the preprint authors should have been able work forwards and calculate conditional probabilities of infection with the two strains given the vaccine status, and how fast those probabilities seem ("seem", given that it won't be a controlled experiment) to go down over time after each dose of the vaccine.
When it comes to journalists, sometimes a story that reveals the key facts if you read all the way to the end has a headline that's alarmist or misleading. (I suspect that editors who want to grab peoples' attention write some of the headlines.) So it's worth distinguishing whether it's just the headline that misleading, or the whole story. Disappointing to see that in this case the story is also misleading.
The cases are strikingly few (speaking to misleading headline/content: how few the cases are should have been a bigger deal) but yes, if we knew both the vaccinated denominator and also whether the cases are related to one another, it could give us an initial sense. Especially if they kept it up! But there were zero cases after two weeks of B.1.351! Can't even calculate! Why wasn't the headline?
I agree with your overall points and conclusions, but the statement "A study that could show if this variant was indeed evading immunity from the Pfizer vaccine would, necessarily, work the other way." appears to be too strong a dismissal. It also seems inconsistent with your other essays in which you rightly criticize the experts who unreasonably demanded too much confirmation before reluctantly releasing their prior biases, e.g. conceding that the virus is transmitted via aerosols. A forward study to address this question is probably impossible at this point, if only for ethical reasons. Imagine hypothetically that a reverse study is conducted at some point in the future and finds that among 10,000 fully vaccinated people, there are 100 breakthrough cases found in a one week period. Suppose further that 95 of these cases are genotyped to a particular variant of concern. Suppose further that variant surveillance shows that 50% of cases in the relevant environment during that week are associated with the variant of concern. (Nearly all of these cases would need to be in unvaccinated or partly vaccinated people in the same environment as the study population, not the study population itself.) Assume correct age matching, household declustering, etc. Would this hypothetical study not be strong evidence that the vaccine were significantly less effective agains this hypothetical variant? Realistically, if a variant emerges with significant vaccine escape, other than strong anecdotal evidence, a hypothetical study like this one is probably the strongest type of controlled quantitative evidence we are likely to get. In reality, of course, the result would be unlikely to be so clean. My point is that we should not preclude the value of a future, properly conducted retrospective study to inform us on this critical question, especially since it's the best we're likely to get.
Can someone please post the link to the preprint? It is probably in some of the misleading news reports, which I didn't read because I could see they could have been produced by carnival barkers.
I think this is more evidence of initial failure to properly analyze data presented and the unnecessary fear that headlines based on limited data create.
Thank you Zeynep. The level of distress and worry being caused by the whole "variants" thing is extraordinary. To pick just one example, a colleague (we're not scientists) fully expects a highly vaccine resistant variant to appear later this year because there were still so many unvaccinated people in the world for the virus to mutate in, meaning that NPI's will have to resume. As a non-scientist I tried the following argument out on him. Does it fly?
Consideration no1: unlike flu, most of our vaccines are extremely effective, so the bar is high. Consideration no2: we have had global spread of the virus for over a year, and the three variants that emerged in that year are, best we can tell, not significantly vaccine escaping: obviously not conclusive, but encouraging in terms of how fast and extremely it mutates. Consideration no3, if a variant were to mutate enough to significantly evade vaccines, it might well change in other ways (eg not be able to spread asymptomatically). My conclusion from those is that the risk of a significantly vaccine-immune CV19 is not obviously greater than the risk of a new flu-type virus altogether, with who knows what characteristics (could be worse, better, more like SARS/MERS, anything). Which is of course a real risk - but was in 2015, and will be in 2025, and in any case is not something that we the general public even now are particularly worrying about, or should worry. Hence my colleague should feel cheerful, assume life will be back to normal in the autumn and leave it to the experts to monitor variants and viruses more generally. Is that fair?
I hate pointing out typos in otherwise great posts, but this one is potentially confusing: "They found that, in the first two weeks following the second dose, there were 8 cases of infection from B.1.351 compared with **[missing data for]**the UK variant B.1.1.7, which is already dominant in Israel at the time of study..."
Wouldn't the most encouraging finding of this particular study, even beyond vaccine effectiveness, be that the South African variant doesn't appear to be spreading any further in Israel but holding steady at 10%? (Which means that even if by some chance it DID cause significantly more breakthrough infections, this would be more than outweighed by its inability to outcompete other strains). I feel like this is one example of the importance of considering and reporting every different piece of good news in a particular set of findings, including relative and contextual ones; not just the obvious data points like number of breakthrough infections. It wouldn't have immediately occurred to me to check out the long-term trend in B.1.351's competitive fitness, even though this is at least as significant as the more absolute measures of its impact.
I had a professor who joked that when he first started out, whenever he read journal articles he would naively start with the intro and discussion sections and read those very thoroughly, dissecting their implications, and then check the methods section after the fact to verify everything. But now he goes right to the methods section and spends all his time reading that, glances at the intro, and then glances at the discussion and immediately says "Oh I see, you're pretending this means X but really it means Y."
Oh crap! ONE. Eight cases versus one. THANK YOU FOR THE CATCH. And sorry about that.
So yes, that's super interesting, no, whether it can outcompete? This doesn't give us a full sense because... it's not clear that the eight are all independent cases: if there are two big clusters there, that's not really telling us that much. But, my initial sense from couple of countries is B.1.1.7 is outcompeting both P1 and this one. B.1.1.7 is also bad news, but often not portrayed in same terrible news because it doesn't have the (widely misunderstood) "immune escape" property (which sounds like it makes vaccines ineffective when that's not really the full case) while it has other terrible, terrible properties like being really competitive, contagious and more severe, ugh. And yeah, I always start with methods section, too. Same lesson. Nothing else makes sense, really, before one understands the methods. I've written papers. We all make claims in intro and discussion. Methods is where reality is really strong, and from everything else flows! If I can't understand what's being said in a methods paper, I tend to get really wary that I am in a position to say much about the paper. Will either seek a colleague/someone else who is in a position to judge what's going on or just decide or be really cautious of my own understanding.
Plus, the paper is properly and heavily caveated. It's an interesting one, but really, really not how it got portrayed.
"Methods is where reality is really strong, and from everything else flows." What a great line! You should totally use that in another article. I guess there's also a way in which everything must flow from theory and background conceptual work to come up with the appropriate method in the first place, but that would be the design of the study whereas everything starts with method for the interpretation, analysis and conclusion (including by the reader).
The irony is that the power of method can be undermined by methodological rituals that privilege a very narrow range of methods, even when they are insufficiently sensitive to the particular question or thing being studied - but then elevate this "method" above all other considerations. This is a big problem in psychology (they like to call it "methodolatry").
This is a really good piece. Thank you as always Zeynep. Immunology 101 is not easy. Then you learn Immunology 101 and you think you got it! Then in your senior level classes you learn all of the exceptions and all of the rules you thought were black and white are not.
But here are some basic black & whites that I remember from 101 that often come to my mind and someone please tell me if I am out in left field.
Why do we need a new flu vaccine every year? My first thought is, the mutation rate of influenza is such that their are a dozen of worrisome variants every year that evade the previous years vaccine. We focus on the top 3 or 4 watching the southern hemisphere and generate a vaccine for those.
Why is the flu vaccine only 30 to 60% affective? Because by the time you choose the target variants which are going strong in the southern hemisphere & make the respective vaccines, the probability exists that one of the other worrisome variants ends up being the more dominate, one that was not selected or the ones selected have further mutated.
Why do we only need a booster for other diseases every so many years like tetanus? I think one reason is that it depends on the rate of how fast or slow the virus or bacterium invade the body. Clostridium tetani as an example probably takes its sweat time in replicating in the body and spreading giving plenty of warning to our defense system. The residual specific memory B cells for example made from previous exposure do not have to be in super high numbers to start its defense, giving time for sufficient B cells to come out in full force..
So, will SARS-CoV-2 generate variants at a similar rate as influenza? I think down the road it is possible we will be watching a few hundred that 1 or a dozen will eventually evade our current vaccines in force today.
Thank you, David. I will not pretend to comment on the immunology part (I like the way Ed Yong at the Atlantic put it: where intuition goes to die!) and maybe someone with expertise on that will step up and comment. With this particular study, just reading the statistics/methods was enough to understand what they were (and were NOT) saying.
I think whether SARS-CoV-2 has reached some sort of optimal fitness peak (with not much room to grow) or if it has a lot bigger landscape remaining to explore is one of those questions we don't (yet) know the answer to, but I think it looks like we would be able to vaccinate pretty well in either case if we put the resources into it, so I'm generally hopeful on the "we can" part but "will we" remains an open question.
This was a helpful example for showing how understanding methods and carefully reading the actual study are so important and can prevent misinterpretation of results. Venting just a little here, but it has been very frustrating discussing post-vaccination behavior with roommates who are heavily engaged with a lot of news sources that have been heavily emphasizing the potential threat of variants. It has been very difficult to acknowledge that there are real concerns there, but that our vaccinations (particularly the mRNA) have shown little reason to panic to this point or fret that the pandemic will never end. And also there seems to be a poor understanding as you've been saying that a small number of breakthrough cases aren't necessarily cause for alarm either.
I'm not offering analysis here, just expressing some exasperation... and also saying thank you!
I share your exasperation! Even as I write those words, I'm reminded that I shouldn't be surprised as every "corner" we turn through this process has been quite difficult - any time we're adjusting our behavior, the social dynamics of navigating the change can be so difficult - especially if you are someone who is sort of in-between the extremes and wanting to make decisions guided at least partially by science and data.
I'm part of a very cautious homeschool pod and it is so hard to discuss potential (slight!) behavior changes for our last two months as some people want to maintain masking between fully vaccinated (and cautious) adults outdoors. As someone is interested in both the science (doesn't get much more low risk than fully vaccinated, cautious, adults outdoors) and social dynamics (some degree of autonomy is needed for any successful group to stay healthy) - this seems like such an easy change to make and yet fear over breakthrough cases and variants and "the data is so new" and you never know.... make for a more cautious end of the year even as we are mostly fully vaccinated adults in our group. Sigh.
Yes, exactly! I feel the same way. It's hard (both as a matter of reaching the right understanding and of carrying that understanding into social situations) to try to adhere to the science, rather than one of the oppositional extreme interpretations of it. Thanks, Katie!
I think we need a medical/scientific equivalent of "snopes" to cover scientific papers, which journalists would consult before printing their stories. Over and over again it's clear that journalists are not up to the task of understanding these papers. Perhaps they could come together to fund such an organization which they could all make use of.
That would be excellent, but could probably provide a lot of work! It's not that great out there.
Given that these news misreports are the issue with the Covid coverage, which is a much smaller set of papers and is the set we are concerned about here *and* is the purview of Snopes itself (news misreporting), I propose that Zeynep approach Snopes with this proposal and offer to recommend some Snopers to do these reviews. Plus the Snopers could submit perhaps shorter versions of the ones Zeynep does herself.
It seems like the preprint authors should have been able work forwards and calculate conditional probabilities of infection with the two strains given the vaccine status, and how fast those probabilities seem ("seem", given that it won't be a controlled experiment) to go down over time after each dose of the vaccine.
When it comes to journalists, sometimes a story that reveals the key facts if you read all the way to the end has a headline that's alarmist or misleading. (I suspect that editors who want to grab peoples' attention write some of the headlines.) So it's worth distinguishing whether it's just the headline that misleading, or the whole story. Disappointing to see that in this case the story is also misleading.
The cases are strikingly few (speaking to misleading headline/content: how few the cases are should have been a bigger deal) but yes, if we knew both the vaccinated denominator and also whether the cases are related to one another, it could give us an initial sense. Especially if they kept it up! But there were zero cases after two weeks of B.1.351! Can't even calculate! Why wasn't the headline?
I agree with your overall points and conclusions, but the statement "A study that could show if this variant was indeed evading immunity from the Pfizer vaccine would, necessarily, work the other way." appears to be too strong a dismissal. It also seems inconsistent with your other essays in which you rightly criticize the experts who unreasonably demanded too much confirmation before reluctantly releasing their prior biases, e.g. conceding that the virus is transmitted via aerosols. A forward study to address this question is probably impossible at this point, if only for ethical reasons. Imagine hypothetically that a reverse study is conducted at some point in the future and finds that among 10,000 fully vaccinated people, there are 100 breakthrough cases found in a one week period. Suppose further that 95 of these cases are genotyped to a particular variant of concern. Suppose further that variant surveillance shows that 50% of cases in the relevant environment during that week are associated with the variant of concern. (Nearly all of these cases would need to be in unvaccinated or partly vaccinated people in the same environment as the study population, not the study population itself.) Assume correct age matching, household declustering, etc. Would this hypothetical study not be strong evidence that the vaccine were significantly less effective agains this hypothetical variant? Realistically, if a variant emerges with significant vaccine escape, other than strong anecdotal evidence, a hypothetical study like this one is probably the strongest type of controlled quantitative evidence we are likely to get. In reality, of course, the result would be unlikely to be so clean. My point is that we should not preclude the value of a future, properly conducted retrospective study to inform us on this critical question, especially since it's the best we're likely to get.
Can someone please post the link to the preprint? It is probably in some of the misleading news reports, which I didn't read because I could see they could have been produced by carnival barkers.
https://www.medrxiv.org/content/10.1101/2021.04.06.21254882v1
Thank you!
A short article from the Virology Blog suggesting that in fact the UK virus is not more virulent than the original Covid virus. https://www.virology.ws/2021/04/15/sars-cov-2-variant-b117-is-not-more-virulent/
I think this is more evidence of initial failure to properly analyze data presented and the unnecessary fear that headlines based on limited data create.
Thank you Zeynep. The level of distress and worry being caused by the whole "variants" thing is extraordinary. To pick just one example, a colleague (we're not scientists) fully expects a highly vaccine resistant variant to appear later this year because there were still so many unvaccinated people in the world for the virus to mutate in, meaning that NPI's will have to resume. As a non-scientist I tried the following argument out on him. Does it fly?
Consideration no1: unlike flu, most of our vaccines are extremely effective, so the bar is high. Consideration no2: we have had global spread of the virus for over a year, and the three variants that emerged in that year are, best we can tell, not significantly vaccine escaping: obviously not conclusive, but encouraging in terms of how fast and extremely it mutates. Consideration no3, if a variant were to mutate enough to significantly evade vaccines, it might well change in other ways (eg not be able to spread asymptomatically). My conclusion from those is that the risk of a significantly vaccine-immune CV19 is not obviously greater than the risk of a new flu-type virus altogether, with who knows what characteristics (could be worse, better, more like SARS/MERS, anything). Which is of course a real risk - but was in 2015, and will be in 2025, and in any case is not something that we the general public even now are particularly worrying about, or should worry. Hence my colleague should feel cheerful, assume life will be back to normal in the autumn and leave it to the experts to monitor variants and viruses more generally. Is that fair?
I hate pointing out typos in otherwise great posts, but this one is potentially confusing: "They found that, in the first two weeks following the second dose, there were 8 cases of infection from B.1.351 compared with **[missing data for]**the UK variant B.1.1.7, which is already dominant in Israel at the time of study..."
Wouldn't the most encouraging finding of this particular study, even beyond vaccine effectiveness, be that the South African variant doesn't appear to be spreading any further in Israel but holding steady at 10%? (Which means that even if by some chance it DID cause significantly more breakthrough infections, this would be more than outweighed by its inability to outcompete other strains). I feel like this is one example of the importance of considering and reporting every different piece of good news in a particular set of findings, including relative and contextual ones; not just the obvious data points like number of breakthrough infections. It wouldn't have immediately occurred to me to check out the long-term trend in B.1.351's competitive fitness, even though this is at least as significant as the more absolute measures of its impact.
I had a professor who joked that when he first started out, whenever he read journal articles he would naively start with the intro and discussion sections and read those very thoroughly, dissecting their implications, and then check the methods section after the fact to verify everything. But now he goes right to the methods section and spends all his time reading that, glances at the intro, and then glances at the discussion and immediately says "Oh I see, you're pretending this means X but really it means Y."
Oh crap! ONE. Eight cases versus one. THANK YOU FOR THE CATCH. And sorry about that.
So yes, that's super interesting, no, whether it can outcompete? This doesn't give us a full sense because... it's not clear that the eight are all independent cases: if there are two big clusters there, that's not really telling us that much. But, my initial sense from couple of countries is B.1.1.7 is outcompeting both P1 and this one. B.1.1.7 is also bad news, but often not portrayed in same terrible news because it doesn't have the (widely misunderstood) "immune escape" property (which sounds like it makes vaccines ineffective when that's not really the full case) while it has other terrible, terrible properties like being really competitive, contagious and more severe, ugh. And yeah, I always start with methods section, too. Same lesson. Nothing else makes sense, really, before one understands the methods. I've written papers. We all make claims in intro and discussion. Methods is where reality is really strong, and from everything else flows! If I can't understand what's being said in a methods paper, I tend to get really wary that I am in a position to say much about the paper. Will either seek a colleague/someone else who is in a position to judge what's going on or just decide or be really cautious of my own understanding.
Plus, the paper is properly and heavily caveated. It's an interesting one, but really, really not how it got portrayed.
"Methods is where reality is really strong, and from everything else flows." What a great line! You should totally use that in another article. I guess there's also a way in which everything must flow from theory and background conceptual work to come up with the appropriate method in the first place, but that would be the design of the study whereas everything starts with method for the interpretation, analysis and conclusion (including by the reader).
The irony is that the power of method can be undermined by methodological rituals that privilege a very narrow range of methods, even when they are insufficiently sensitive to the particular question or thing being studied - but then elevate this "method" above all other considerations. This is a big problem in psychology (they like to call it "methodolatry").
This is a really good piece. Thank you as always Zeynep. Immunology 101 is not easy. Then you learn Immunology 101 and you think you got it! Then in your senior level classes you learn all of the exceptions and all of the rules you thought were black and white are not.
If you are a master chess or Go player, you get the idea when you get pounded by AI. https://www.youtube.com/watch?v=WXuK6gekU1Y
But here are some basic black & whites that I remember from 101 that often come to my mind and someone please tell me if I am out in left field.
Why do we need a new flu vaccine every year? My first thought is, the mutation rate of influenza is such that their are a dozen of worrisome variants every year that evade the previous years vaccine. We focus on the top 3 or 4 watching the southern hemisphere and generate a vaccine for those.
Why is the flu vaccine only 30 to 60% affective? Because by the time you choose the target variants which are going strong in the southern hemisphere & make the respective vaccines, the probability exists that one of the other worrisome variants ends up being the more dominate, one that was not selected or the ones selected have further mutated.
Why do we only need a booster for other diseases every so many years like tetanus? I think one reason is that it depends on the rate of how fast or slow the virus or bacterium invade the body. Clostridium tetani as an example probably takes its sweat time in replicating in the body and spreading giving plenty of warning to our defense system. The residual specific memory B cells for example made from previous exposure do not have to be in super high numbers to start its defense, giving time for sufficient B cells to come out in full force..
So, will SARS-CoV-2 generate variants at a similar rate as influenza? I think down the road it is possible we will be watching a few hundred that 1 or a dozen will eventually evade our current vaccines in force today.
Thank you, David. I will not pretend to comment on the immunology part (I like the way Ed Yong at the Atlantic put it: where intuition goes to die!) and maybe someone with expertise on that will step up and comment. With this particular study, just reading the statistics/methods was enough to understand what they were (and were NOT) saying.
I think whether SARS-CoV-2 has reached some sort of optimal fitness peak (with not much room to grow) or if it has a lot bigger landscape remaining to explore is one of those questions we don't (yet) know the answer to, but I think it looks like we would be able to vaccinate pretty well in either case if we put the resources into it, so I'm generally hopeful on the "we can" part but "will we" remains an open question.