This makes me think that basic Bayesian reasoning needs to be taught early. People argue as if the various factors (location, properties of previous new viruses,...) are all on different planes, so you just pick your favorite factor and ignore the others. But Bayes provides an obvious systematic way to combine different pieces of evidence, whether they point the same way or opposite ways. Most of the chances for non-lab crossover would occur elsewhere, so we can rule them out. In this case, with Wuhan having a little less than 1% of China's population and not being located in a viral hot spot, you'd take whatever odds you had based on other factors and multiple the Lab/Non-Lab odds ratio by ~100. That's not the ~infinite factor Jon Stewart seems to use but it's also not an irrelevant coincidence.
What would those other odds, not involving location, be? Some people say that there are so many more chances for nature to make new combinations that they are less than 1/100. Others say the lab was a good place for different strains to recombine, so they are more than 1/100. I don't know enough to comment on that. But including the location in the odds calculation is the easiest part to get very roughly right.
As you say, the name-calling adds nothing to the estimate!
Agreed. To assess the strength of the evidence, the two probabilities we care about are:
p(outbreak near lab | natural spillover)
p(outbreak near lab | lab leak from that lab)
I assume that many areas on Earth have some (small) probability of natural coronavirus spillover. I also assume that some of the areas with higher probability of spillover have coronavirus research labs near them.
...Which means that when an outbreak happens, there are many places it could happen that don't have a lab nearby. Sure, some of the more probable locations do have labs, but there must still be a lot of probability mass elsewhere.
Meanwhile, if a coronavirus leaks from a lab, there's a good chance the outbreak will start nearby (or in a place that is readily traceable to the lab, say if samples were shipped somewhere else).
So...
p(outbreak near lab | natural spillover) doesn't seem to be terribly high.
p(outbreak near lab | lab leak from that lab) seems much higher.
Which means the likelihood ratio might be 2x, 3x, or more in favor of the lab leak hypothesis. Of course there are priors and other evidence to take into account, but the location of the outbreak is literally strong evidence in favor of the virus leaking from the lab.
The Washington Post article that I cited above raises a new host of possibilities and I highly recommend it. I am not going to spoil the experience of reading it.
What's our excuse? Probably the social and psychological rewards of the sneering, poorly-thought-through Twitter dunk. The platform makes it so easy and satisfying to come up with a snarky response that sounds intellectually superior, and the culture that has grown up around the platform has normalized those kinds of responses so thoroughly, that even intelligent, well-informed people like Koerth yield to temptation. I've seen it in political as well as scientific disputes; I've seen the smartest people I know retweet dunks that are easily seen, on non-confirmation-biased reflection, to not really answer the argument they're dunking on. It's more evidence that Twitter culture is undermining both our cognition and our moral sense.
That behavior was a mistake, to the extent it was true (don't "we" me). I did dismiss Trumps accusations that the virus was a manufactured bio weapon. I also dismissed China's accusations that our US soldiers brought it in during the military games. The press appeared to lump every other non-zoonotic possibility into the same conspiracy barrel.
Seemed like the story was dead, as far as broadcast news goes, until Politico stuck their neck out in January.
Thank you for focusing on the relevance of lab location. As a layperson, I've never understood why lab location was often dismissed. I see virologists stating that the chance of the pandemic having lab origins is "less than 1% for a lab leak of a natural isolate" and I wonder how on earth they calculate such a low figure. https://twitter.com/florian_krammer/status/1397880781675053057
Before the pandemic, both lab and natural scenarios in Wuhan would have been described as extremely unlikely. They're both very low probability events, but we know that one of them has since happened.
My back-of-an-envelope estimates: 1. For lab-leak accident, outbreak of 1,000 or more coronavirus infections from WIV might happen once in 500 to 50,000 years. 2. For natural spillover in Wuhan, outbreak of 1,000 or more coronavirus infections might happen once in 1,000 to 14,000 years. So both scenarios are very unlikely, but to me neither seems extremely unlikely compared to the other.
I looked at the lab risk assessment for the infectious disease labs at Boston University, which says 1,000+ infections of 1918 H1N1 flu might happen once in 10,000–1 million years. We could assume the same risk for the Wuhan labs and a SARS2-like virus. But if the WIV was doing coronavirus work in BSL2 and allegedly risky virus collecting then the risk of a coronavirus outbreak might be far greater than once in 10,000–1 million years. Could it be as high as once in 500 to 50,000 years? The Boston labs risk assessment summary is at: https://www.bu.edu/neidl/files/2014/07/Final-Supplementary-Risk-Assessment-7_2012-Readers-Guide.pdf
For natural spillover in Wuhan (once in 1,000 to 14,000 years): as a starting point I assumed five global outbreaks per century of 1,000+ infections of novel coronavirus. If I assume the chances of an outbreak in Wuhan are the same as everywhere else then the chances are once in 14,000 years. But coronavirus outbreaks are more likely in region adjacent to Wuhan, more likely to spread in big city, more likely detected in large modern city — so I might arbitrarily assign a lower range of once in 1,000 years. Apologies for the back-of-an-envelope approach although I’m not sure some virologists are doing anything more sophisticated when they calculate the chances of a lab-leak are less than 1%.
But still, wouldn't you have to factor into the probability the reasonable number of years that the WIV could possibly exist? So, let's say 50 years for life of lab.
So instead of a one in 500 chance at the high end, your 50 years only give you one tenth of a shot. So now you are at 1 in 5000 on the high probability end?
Yes, the expected operating life of the lab should definitely be factored in. My layperson's understanding is that if an incident might happen once in 500 years then the chance of it happening in any one year of the lab's operation is 0.2%, and if the lab's expected operating life is 50 years then the chance of it happening over the lifetime of the lab might be about 9.5%.
From the Boston labs risk assessment, I was struck by how difficult it is to estimate the likelihood of low-probability events — the report ends up with wide ranges like "might happen once in 10,000–1 million years". But the fact the Boston labs risk assessment exists shows it's possible to at least try to estimate the risk. I'm surprised that few people who do probability for a living seem to have had a go at comparing the likelihood of 1. a coronavirus outbreak (1,000+ cases) from a coronavirus research lab versus 2. a coronavirus outbreak (1,000+ cases) occurring naturally in Wuhan.
I would just say one criticism... I wish you wouldn't use the word "proof" at all in a conversation like this one. There's no proof to be had outside of math and deductive logic, and letting such binary terms into the conversation just lets people ignore you when you say "there's strong evidence of X" and they say "aha, but not proof!"
I just returned from the Chapel Hill Farmers Market before reading your post. Maybe I missed something, but did not see any bats for sale. Not even frozen bat dogs for Summer BBQ. So why is there a corona virus lab at UNC?
"Add the fact that the known first superspreader event (which we now know likely wasn’t where the virus originated from but just the first known amplifying event)."
What is meant by this qualification? I don't think it's been widely suggested that SARS-CoV-2 evolved or was created by recombination inside animals after they made it to the market, just that the market(s) is the probable location of the spillover event to humans. There is evidence of different virus lineages, some of which do not correspond with the Huanan market, but the temporality of the emergence is not fully clear- I think the notion of the spillover occurring elsewhere has not been established as "likely".
Please let's not focus on two origin possibilities in Wuhan. There are so many more.
The amplifying event qualification means that the virus may have developed outside Wuhan, and we became aware of it in Wuhan. There were many gatheings of large numbers of people in Wuhan during the period in question. If there is going to be a pandemic, Wuhan was a likely place: a large city with lots of large gatherings of people from everywhere in the world at events such as...there were lots of them including the military one.
Sigh, I did not want to reveal what I found in the WaPo article, and maybe you will find more possibilities, but here is one possibility I found in it:
What if a not-very-infectious bat coronavirus was around for a number of months or even years, perhaps carried to countries that have Chinese seasonal or immigrant workers, and bounced around between the two countries (maybe other countries as well) and caused some deaths with symptoms similar to the flu? What if it became more transmissible to humans (sort of like the Alpha, Beta, Gamma Delta variants) and was identified in Wuhan during a major flu season because epidemiologists there distinguished it from the flu?
What if China did not have a good public health system? What if, for example, it took off in the U.S. where we do not have a good public health service and we discounted it as a really bad flu until, maybe it lasted into the summer and we said ... "Maybe something is going on here?" Or maybe we didn't find it at all and eventually some place with a better public health system (such as China) identified it.
I think another major factor is that people, even smart ones, take very strong cues from people they trust about which ideas are believable. (It's pretty hard not to, since science is so complicated.) And the general message for the past year about the lab leak hypothesis from major trustworthy-ish sources has been "this is nonsense and we totally know it's false."
The point is, there's nothing racist about protesting Chinese human rights violations, just as there's nothing antisemitic about protesting Isreal treat Palestinians as third-class people.
Conflating the two is a staple of pseudo-liberal simplification.
This makes me think that basic Bayesian reasoning needs to be taught early. People argue as if the various factors (location, properties of previous new viruses,...) are all on different planes, so you just pick your favorite factor and ignore the others. But Bayes provides an obvious systematic way to combine different pieces of evidence, whether they point the same way or opposite ways. Most of the chances for non-lab crossover would occur elsewhere, so we can rule them out. In this case, with Wuhan having a little less than 1% of China's population and not being located in a viral hot spot, you'd take whatever odds you had based on other factors and multiple the Lab/Non-Lab odds ratio by ~100. That's not the ~infinite factor Jon Stewart seems to use but it's also not an irrelevant coincidence.
What would those other odds, not involving location, be? Some people say that there are so many more chances for nature to make new combinations that they are less than 1/100. Others say the lab was a good place for different strains to recombine, so they are more than 1/100. I don't know enough to comment on that. But including the location in the odds calculation is the easiest part to get very roughly right.
As you say, the name-calling adds nothing to the estimate!
Agreed. To assess the strength of the evidence, the two probabilities we care about are:
p(outbreak near lab | natural spillover)
p(outbreak near lab | lab leak from that lab)
I assume that many areas on Earth have some (small) probability of natural coronavirus spillover. I also assume that some of the areas with higher probability of spillover have coronavirus research labs near them.
...Which means that when an outbreak happens, there are many places it could happen that don't have a lab nearby. Sure, some of the more probable locations do have labs, but there must still be a lot of probability mass elsewhere.
Meanwhile, if a coronavirus leaks from a lab, there's a good chance the outbreak will start nearby (or in a place that is readily traceable to the lab, say if samples were shipped somewhere else).
So...
p(outbreak near lab | natural spillover) doesn't seem to be terribly high.
p(outbreak near lab | lab leak from that lab) seems much higher.
Which means the likelihood ratio might be 2x, 3x, or more in favor of the lab leak hypothesis. Of course there are priors and other evidence to take into account, but the location of the outbreak is literally strong evidence in favor of the virus leaking from the lab.
The Washington Post article that I cited above raises a new host of possibilities and I highly recommend it. I am not going to spoil the experience of reading it.
What's our excuse? Probably the social and psychological rewards of the sneering, poorly-thought-through Twitter dunk. The platform makes it so easy and satisfying to come up with a snarky response that sounds intellectually superior, and the culture that has grown up around the platform has normalized those kinds of responses so thoroughly, that even intelligent, well-informed people like Koerth yield to temptation. I've seen it in political as well as scientific disputes; I've seen the smartest people I know retweet dunks that are easily seen, on non-confirmation-biased reflection, to not really answer the argument they're dunking on. It's more evidence that Twitter culture is undermining both our cognition and our moral sense.
We (liberals) let Trump break our brains and bring us down to his level.
That behavior was a mistake, to the extent it was true (don't "we" me). I did dismiss Trumps accusations that the virus was a manufactured bio weapon. I also dismissed China's accusations that our US soldiers brought it in during the military games. The press appeared to lump every other non-zoonotic possibility into the same conspiracy barrel.
Seemed like the story was dead, as far as broadcast news goes, until Politico stuck their neck out in January.
So no we have come around to groupthink.
Thank you for focusing on the relevance of lab location. As a layperson, I've never understood why lab location was often dismissed. I see virologists stating that the chance of the pandemic having lab origins is "less than 1% for a lab leak of a natural isolate" and I wonder how on earth they calculate such a low figure. https://twitter.com/florian_krammer/status/1397880781675053057
Before the pandemic, both lab and natural scenarios in Wuhan would have been described as extremely unlikely. They're both very low probability events, but we know that one of them has since happened.
My back-of-an-envelope estimates: 1. For lab-leak accident, outbreak of 1,000 or more coronavirus infections from WIV might happen once in 500 to 50,000 years. 2. For natural spillover in Wuhan, outbreak of 1,000 or more coronavirus infections might happen once in 1,000 to 14,000 years. So both scenarios are very unlikely, but to me neither seems extremely unlikely compared to the other.
I looked at the lab risk assessment for the infectious disease labs at Boston University, which says 1,000+ infections of 1918 H1N1 flu might happen once in 10,000–1 million years. We could assume the same risk for the Wuhan labs and a SARS2-like virus. But if the WIV was doing coronavirus work in BSL2 and allegedly risky virus collecting then the risk of a coronavirus outbreak might be far greater than once in 10,000–1 million years. Could it be as high as once in 500 to 50,000 years? The Boston labs risk assessment summary is at: https://www.bu.edu/neidl/files/2014/07/Final-Supplementary-Risk-Assessment-7_2012-Readers-Guide.pdf
For natural spillover in Wuhan (once in 1,000 to 14,000 years): as a starting point I assumed five global outbreaks per century of 1,000+ infections of novel coronavirus. If I assume the chances of an outbreak in Wuhan are the same as everywhere else then the chances are once in 14,000 years. But coronavirus outbreaks are more likely in region adjacent to Wuhan, more likely to spread in big city, more likely detected in large modern city — so I might arbitrarily assign a lower range of once in 1,000 years. Apologies for the back-of-an-envelope approach although I’m not sure some virologists are doing anything more sophisticated when they calculate the chances of a lab-leak are less than 1%.
My head is spinning.
But still, wouldn't you have to factor into the probability the reasonable number of years that the WIV could possibly exist? So, let's say 50 years for life of lab.
So instead of a one in 500 chance at the high end, your 50 years only give you one tenth of a shot. So now you are at 1 in 5000 on the high probability end?
Yes, the expected operating life of the lab should definitely be factored in. My layperson's understanding is that if an incident might happen once in 500 years then the chance of it happening in any one year of the lab's operation is 0.2%, and if the lab's expected operating life is 50 years then the chance of it happening over the lifetime of the lab might be about 9.5%.
From the Boston labs risk assessment, I was struck by how difficult it is to estimate the likelihood of low-probability events — the report ends up with wide ranges like "might happen once in 10,000–1 million years". But the fact the Boston labs risk assessment exists shows it's possible to at least try to estimate the risk. I'm surprised that few people who do probability for a living seem to have had a go at comparing the likelihood of 1. a coronavirus outbreak (1,000+ cases) from a coronavirus research lab versus 2. a coronavirus outbreak (1,000+ cases) occurring naturally in Wuhan.
You have to start with the risk of this happening at all. And yet it did.
Love the writing, Zeynep!
I would just say one criticism... I wish you wouldn't use the word "proof" at all in a conversation like this one. There's no proof to be had outside of math and deductive logic, and letting such binary terms into the conversation just lets people ignore you when you say "there's strong evidence of X" and they say "aha, but not proof!"
I almost didn't read this article because I expected "more of the same," but I found it to be extremely informative: https://www.sciencemag.org/sites/default/files/Shi%20Zhengli%20Q%26A.pdf
This is great and I particularly like the tweets from Maggie Koerth. It reminded me of a cartoon years ago in the Reader's Digest:
Caller to Highway department: "I want you to take down the Deer Crossing sign in front of my house.
Highway Department: "Why."
Caller: "I don't want the deer to cross there any more!"
We as a society used to be able to figure these things out...
Also I had not sen the comments on the Science Magazine article (https://www.sciencemag.org/sites/default/files/Shi%20Zhengli%20Q%26A.pdf) and I would like to have a link to the article.
I just returned from the Chapel Hill Farmers Market before reading your post. Maybe I missed something, but did not see any bats for sale. Not even frozen bat dogs for Summer BBQ. So why is there a corona virus lab at UNC?
I hope you didn't see any volcano monitoring stations.
It seems like there's as much political bias as confirmation bias.
"Add the fact that the known first superspreader event (which we now know likely wasn’t where the virus originated from but just the first known amplifying event)."
What is meant by this qualification? I don't think it's been widely suggested that SARS-CoV-2 evolved or was created by recombination inside animals after they made it to the market, just that the market(s) is the probable location of the spillover event to humans. There is evidence of different virus lineages, some of which do not correspond with the Huanan market, but the temporality of the emergence is not fully clear- I think the notion of the spillover occurring elsewhere has not been established as "likely".
This remains the leading theory among the non-lab-leak-inclined scientific community- a population of infected animals, perhaps from a single farm, was distributed to multiple markets throughout Wuhan. See figure 1 here: https://virological.org/t/early-appearance-of-two-distinct-genomic-lineages-of-sars-cov-2-in-different-wuhan-wildlife-markets-suggests-sars-cov-2-has-a-natural-origin/691 Despite official denials and or cover-up, there has been some good research into the conditions and illegal contents of the wet markets: https://www.nature.com/articles/s41598-021-91470-2
Please let's not focus on two origin possibilities in Wuhan. There are so many more.
The amplifying event qualification means that the virus may have developed outside Wuhan, and we became aware of it in Wuhan. There were many gatheings of large numbers of people in Wuhan during the period in question. If there is going to be a pandemic, Wuhan was a likely place: a large city with lots of large gatherings of people from everywhere in the world at events such as...there were lots of them including the military one.
Sigh, I did not want to reveal what I found in the WaPo article, and maybe you will find more possibilities, but here is one possibility I found in it:
What if a not-very-infectious bat coronavirus was around for a number of months or even years, perhaps carried to countries that have Chinese seasonal or immigrant workers, and bounced around between the two countries (maybe other countries as well) and caused some deaths with symptoms similar to the flu? What if it became more transmissible to humans (sort of like the Alpha, Beta, Gamma Delta variants) and was identified in Wuhan during a major flu season because epidemiologists there distinguished it from the flu?
What if China did not have a good public health system? What if, for example, it took off in the U.S. where we do not have a good public health service and we discounted it as a really bad flu until, maybe it lasted into the summer and we said ... "Maybe something is going on here?" Or maybe we didn't find it at all and eventually some place with a better public health system (such as China) identified it.
I agree that there's skittishness.
I think another major factor is that people, even smart ones, take very strong cues from people they trust about which ideas are believable. (It's pretty hard not to, since science is so complicated.) And the general message for the past year about the lab leak hypothesis from major trustworthy-ish sources has been "this is nonsense and we totally know it's false."
The point is, there's nothing racist about protesting Chinese human rights violations, just as there's nothing antisemitic about protesting Isreal treat Palestinians as third-class people.
Conflating the two is a staple of pseudo-liberal simplification.