I would say this lesson is just as important in navigating the day to day online information landscape as it is navigating information related to the pandemic. We desperately need digital/media literacy.
Sometimes there's a difficult tradeoff between conveying important detail on the one hand and remaining accessible/interesting to your audience on the other. This wouldn't seem to be one of those times: this would have been a perfect opportunity for the NYT to introduce these concepts and demonstrate the analysis in context. Even if their readers wouldn't have been ready for it a year ago, they are now. (And I'd claim NYT reporting often does rise to that level.) Too bad you weren't there to review @zeynep!
Great piece. The pedant in me won't let you off on one point: "the average 83-year-old has a life expectancy of seven to eight more years, which means half of them live even longer". Those life expectancies are means not medians and life expectancy is skewed, no less than half will live longer than the life expectancy. Doesn't affect any of your arguments of course.
This is a great example of good news that is so easy to read or spin as bad news. But I feel a bit conflicted on this one. You write: "Facts need the right context so that they can turn into information that’s more useful to us...People want to understand the details. But facts don’t just float around by themselves, they are pieces of a puzzle." Very true - I'm reminded of the crossword puzzle metaphor Susan Haack used to illustrate her "foundherentist" pragmatist approach to evidence.
Yet I can't help feeling you didn't go quite far enough in speaking to readers' real concerns and what may be most salient to them. From some perspectives, the apples-to-apples comparisons of vaccinated vs. non-vaccinated and continued evidence of efficacy in this setting vs. clinical trials is beside the point, if the relative numbers of infected and symptomatic still feel too high for comfort in their own right. Even if the headline were worded like Bill Hanage's tweet, once you see that even 10% of those fully vaccinated got infected, 6.3% had symptoms and around 1% actually died (higher than IFR in the general population!), that sounds very discouraging on its face. Sure, it's a nursing home and overdispersion etc., but even told this, most people don't have a ready reference point for how that alters things quantitatively; in the meantime what they know is "10%/6.3%/1%." And overdispersion works both ways: that means if you DO find yourself in a cluster, you're screwed! (If-then conditional scenarios can feel even scarier than flat risk, by emphasizing unpredictability). Seen this way, Hanage's cheery take may come off as obfuscation. To understand why it's truly good news you need a Zeynep-style full-length narrative explication like the one above, and even then it might take several times for all this context to sink in.
So I feel like proper framing and contextualization of facts as detailed here is not enough. There also must be a meta-framing and meta-contextualization of that framing and context, which directly and empathetically addresses the saliency of certain aspects of the data (the visceral power of numbers, underestimating effect of the nursing home setting, overdisperson as a scary "if-then" not a reassuring "only-if"), and where this might persist *even when* it has been presented as perfectly as possible. It may be necessary to literally walk people through the interpretive process as a kind of teaching practice, much as your own posts and articles attempt to do; not just responsibly present information in the best way possible and leave readers to misinterpret on their own. This too is part of the meta-epistemology you're aiming at, I think.
Reading your excellent article and the NYT piece, my main takeaway is that careful personal behavior is still critical. Five percent is one chance out of 20 -- even if hospitalization is not required. those odds are not good enough for me. I'm no longer living like a hermit, but I don't think this is over. I think the reality that on a world basis we are still so lost is an important context for understanding the protective value of the vaccine. It is a big world. Evolution happens. We are congratulating ourselves (with some justification) while the world as a whole is in deep trouble.
I'm wondering what this community thinks of headlines /stories like this (link below). The headline starts with rare and expected, which is great. But is the very use of this as a headline, and making the rare and expected part a qualifier rather than the main emphasis, problematic?
There's one piece from the original article that seems to contradict parts of your take:
“Vaccine was 86.5% protective against symptomatic illness among residents and 87.1% protective among HCP.”
The vaccine effectiveness appears to have been identical between the frail residents and the more able employees. This was, to me, the most surprising thing. It seems to contradict the discussions about lower vaccine effectiveness in the elderly.
Your mentioning the small sample set is key. Also that many residents were immunocompromised.
One other piece of the puzzle here is selection bias. An outbreak at a nursing home which is small and contained won't get news coverage. So when the vaccine efficacy is high, news coverage is low. This makes it expected that the selective picking of one example of an outbreak will show lower efficacy.
Thank you so much for this thoughtful and informative analysis. I wish you wrote every single day!! Someday I hope you will take on what I consider herd-mentality when it comes to reporting on variants:
Scientists: ‘variant needs to be watch carefully to see if it’s more contagious’ ; Media: ‘variant is more contagious’; Politician: ‘close the airports’.
If you were to turn your eye to variants we’d be all ears. Is the “variant scare” evidence-based science or is it more similar to our fixation with fomites? Just because we have DNA sequencing technology doesn’t mean we understand what it MEANS so how does Founder Effect come into play? How about cluster transmission and super-spreading events which you’ve written so well about?
Has anyone seen post-trial statistics on breakthrough illness among those who got the J and J vaccine? Curious how the lower efficacy numbers (of debatable importance) have turned out after millions of doses were given.
My one, tangential, disagreement is that primary focus of the NYT article is on the consequences of having some nursing home healthcare workers unvaccinated. The reporter's discussion of vaccine performance seemed to be a distraction to her main purpose. This article https://www.nytimes.com/2021/04/21/health/vaccine-nursing-homes-infections.html and the Boston Globe article referred by jdnym fall into the half full category. And maybe there is a learning curve that is kicking in
Wonderfully laid out how facts in their context tell a wonderful story. But how to convey to wide audiences when negative news elicit stronger psychophysiological response, which generate clicks which generate revenue...
The English editor within me couldn't help noticing a small number of places where editing is needed, particularly since this will be open to the public. Hope you don't mind my pointing this out.
I would say this lesson is just as important in navigating the day to day online information landscape as it is navigating information related to the pandemic. We desperately need digital/media literacy.
Sometimes there's a difficult tradeoff between conveying important detail on the one hand and remaining accessible/interesting to your audience on the other. This wouldn't seem to be one of those times: this would have been a perfect opportunity for the NYT to introduce these concepts and demonstrate the analysis in context. Even if their readers wouldn't have been ready for it a year ago, they are now. (And I'd claim NYT reporting often does rise to that level.) Too bad you weren't there to review @zeynep!
Great piece. The pedant in me won't let you off on one point: "the average 83-year-old has a life expectancy of seven to eight more years, which means half of them live even longer". Those life expectancies are means not medians and life expectancy is skewed, no less than half will live longer than the life expectancy. Doesn't affect any of your arguments of course.
This is a great example of good news that is so easy to read or spin as bad news. But I feel a bit conflicted on this one. You write: "Facts need the right context so that they can turn into information that’s more useful to us...People want to understand the details. But facts don’t just float around by themselves, they are pieces of a puzzle." Very true - I'm reminded of the crossword puzzle metaphor Susan Haack used to illustrate her "foundherentist" pragmatist approach to evidence.
Yet I can't help feeling you didn't go quite far enough in speaking to readers' real concerns and what may be most salient to them. From some perspectives, the apples-to-apples comparisons of vaccinated vs. non-vaccinated and continued evidence of efficacy in this setting vs. clinical trials is beside the point, if the relative numbers of infected and symptomatic still feel too high for comfort in their own right. Even if the headline were worded like Bill Hanage's tweet, once you see that even 10% of those fully vaccinated got infected, 6.3% had symptoms and around 1% actually died (higher than IFR in the general population!), that sounds very discouraging on its face. Sure, it's a nursing home and overdispersion etc., but even told this, most people don't have a ready reference point for how that alters things quantitatively; in the meantime what they know is "10%/6.3%/1%." And overdispersion works both ways: that means if you DO find yourself in a cluster, you're screwed! (If-then conditional scenarios can feel even scarier than flat risk, by emphasizing unpredictability). Seen this way, Hanage's cheery take may come off as obfuscation. To understand why it's truly good news you need a Zeynep-style full-length narrative explication like the one above, and even then it might take several times for all this context to sink in.
So I feel like proper framing and contextualization of facts as detailed here is not enough. There also must be a meta-framing and meta-contextualization of that framing and context, which directly and empathetically addresses the saliency of certain aspects of the data (the visceral power of numbers, underestimating effect of the nursing home setting, overdisperson as a scary "if-then" not a reassuring "only-if"), and where this might persist *even when* it has been presented as perfectly as possible. It may be necessary to literally walk people through the interpretive process as a kind of teaching practice, much as your own posts and articles attempt to do; not just responsibly present information in the best way possible and leave readers to misinterpret on their own. This too is part of the meta-epistemology you're aiming at, I think.
Reading your excellent article and the NYT piece, my main takeaway is that careful personal behavior is still critical. Five percent is one chance out of 20 -- even if hospitalization is not required. those odds are not good enough for me. I'm no longer living like a hermit, but I don't think this is over. I think the reality that on a world basis we are still so lost is an important context for understanding the protective value of the vaccine. It is a big world. Evolution happens. We are congratulating ourselves (with some justification) while the world as a whole is in deep trouble.
I'm wondering what this community thinks of headlines /stories like this (link below). The headline starts with rare and expected, which is great. But is the very use of this as a headline, and making the rare and expected part a qualifier rather than the main emphasis, problematic?
https://www.bostonglobe.com/2021/04/21/nation/its-rare-some-fully-vaccinated-people-are-catching-covid-19/
There's one piece from the original article that seems to contradict parts of your take:
“Vaccine was 86.5% protective against symptomatic illness among residents and 87.1% protective among HCP.”
The vaccine effectiveness appears to have been identical between the frail residents and the more able employees. This was, to me, the most surprising thing. It seems to contradict the discussions about lower vaccine effectiveness in the elderly.
Your mentioning the small sample set is key. Also that many residents were immunocompromised.
One other piece of the puzzle here is selection bias. An outbreak at a nursing home which is small and contained won't get news coverage. So when the vaccine efficacy is high, news coverage is low. This makes it expected that the selective picking of one example of an outbreak will show lower efficacy.
Thank you so much for this thoughtful and informative analysis. I wish you wrote every single day!! Someday I hope you will take on what I consider herd-mentality when it comes to reporting on variants:
Scientists: ‘variant needs to be watch carefully to see if it’s more contagious’ ; Media: ‘variant is more contagious’; Politician: ‘close the airports’.
If you were to turn your eye to variants we’d be all ears. Is the “variant scare” evidence-based science or is it more similar to our fixation with fomites? Just because we have DNA sequencing technology doesn’t mean we understand what it MEANS so how does Founder Effect come into play? How about cluster transmission and super-spreading events which you’ve written so well about?
Please clear this up for us!
Has anyone seen post-trial statistics on breakthrough illness among those who got the J and J vaccine? Curious how the lower efficacy numbers (of debatable importance) have turned out after millions of doses were given.
As always, a comprehensive and convincing essay.
My one, tangential, disagreement is that primary focus of the NYT article is on the consequences of having some nursing home healthcare workers unvaccinated. The reporter's discussion of vaccine performance seemed to be a distraction to her main purpose. This article https://www.nytimes.com/2021/04/21/health/vaccine-nursing-homes-infections.html and the Boston Globe article referred by jdnym fall into the half full category. And maybe there is a learning curve that is kicking in
Wonderfully laid out how facts in their context tell a wonderful story. But how to convey to wide audiences when negative news elicit stronger psychophysiological response, which generate clicks which generate revenue...
The English editor within me couldn't help noticing a small number of places where editing is needed, particularly since this will be open to the public. Hope you don't mind my pointing this out.