Root Privileges

Science

A non-self-correcting science

While I’m all for a bit of triumphalism when some component of conventional publication vis-à-vis scientific research – like pre-publication anonymous peer review – fails, and fails publicly, I spotted an article in The Conversation earlier today that I thought crossed a line (and not in the way you think). In this article, headlined ‘Retractions and controversies over coronavirus research show that the process of science is working as it should’, the author writes:

Some people are viewing the retractions [by The Lancet and the New England Journal of Medicine] as an indictment of the scientific process. Certainly, the overturning of these papers is bad news, and there is plenty of blame to go around. But despite these short-term setbacks, the scrutiny and subsequent correction of the papers actually show that science is working. Reporting of the pandemic is allowing people to see, many for the first time, the messy business of scientific progress.
The retraction of the hydroxychloroquine paper … drew immediate attention not only because it placed science in a bad light, but also because President Trump had touted the drug as an effective treatment for COVID-19 despite the lack of strong evidence. Responses in the media were harsh. … [Their] headlines may have [had] merit, but perspective is also needed. Retractions are rare – only about 0.04% of published papers are withdrawn – but scrutiny, update and correction are common. It is how science is supposed to work, and it is happening in all areas of research relating to SARS-CoV-2.

If you ask me, this is not science working as it should. This is the journals that published the papers discovering that the mechanisms they’d adopted that they’d said would filter fraudulent papers letting fraudulent papers slip through.

But by the author’s logic, “this is science working as it should” would encompass any mistake that’s later discovered, followed by suitable corrective action. This is neither here nor there – and more importantly it allows broken processes to be subsumed under the logic’s all-encompassing benevolence. If this is scientific publishing as it should be, we wouldn’t have to think deeply about how we can fix anonymous pre-publication peer-review because it wouldn’t be broken. However, we know in reality that it is.

If anything, by advancing his argument, the author has cleverly pressed an argumentative tack that supporters of more progressive scientific publishing models in the service of preserving the status quo. Instead, we need to acknowledge that an important part of science, called science publishing, has evolved into a flawed creature – so that we can set about bending the moral arc towards fixing it. (We already know that if we don’t acknowledge it, we won’t fix it.)

Jul 8, 2020
Citations and media coverage

According to a press release accompanying a just-published study in PLOS ONE:

Highly cited papers also tend to receive more media attention, although the cause of the association is unclear.

One reason I can think of is a confounding factor that serves as the hidden cause of both phenomena. Discoverability matters just as much as the quality of a paper, and conventional journals implicated in the sustenance of notions like ‘prestige’ (Nature, Science, Cell, The Lancet, etc.) have been known to prefer more sensational positive results. And among researchers that still value publishing in these journals, these papers are more noticed, which leads to a ‘buzz’ that a reporter can pick up on.

Second, sensational results also easily lend themselves to sensational stories in the press, which has often been addicted to the same ‘positivity bias’ that the scientific literature harboured for many decades. In effect, highly cited papers are simply highly visible, and highly visibilised, papers – both to other scientists and journalists.

The press release continues:

The authors add: “Results from this study confirm the idea that media attention given to scientific research is strongly related to scientific citations for that same research. These results can inform scientists who are considering using popular media to increase awareness concerning their work, both within and outside the scientific community.”

I’m not sure what this comment means (I haven’t gone through the paper and it’s possible the paper’s authors discuss this in more detail), but there is already evidence that studies for which preprints are available receive more citations than those published behind a paywall. So perhaps scientists expecting more media coverage of their work should simply make their research more accessible. (It’s also a testament to the extent to which the methods of ‘conventional’ publishers – including concepts like ‘reader pays’ and the journal impact factor, accentuated by notions like ‘prestige’ – have become entrenched that this common-sensical solution is not so common sense.)

On the flip side, journalists also need to be weaned away from ‘top’ journals – I receive a significantly higher number of pitches offering to cover papers published in Nature journals – and retrained to spot interesting results published in less well-known journals as well as, on a slightly separate note, to situate the results of one study in a larger context instead of hyper-focusing on one context-limited set of results.

The work seems interesting, perhaps one of you will like to give it a comb.

Jul 6, 2020
Redeeming art v. redeeming science

Recently, someone shared the cover of a soon to be released book, entitled The Physics of Climate Change, authored by Lawrence M. Krauss and expressed excitement about the book’s impending publication and the prospect of their reading it. I instinctively responded that I would be actively boycotting the book after the sexual harassment allegations against Krauss plus his ties with Jeffrey Epstein. I didn’t, and don’t, wish to consume his scholarship.

Now, I don’t think that facts alone can be redemptive – that if a book’s contents are right, as ascertained through dispassionate tests of verification, we get to ignore questions about whether the contents are good. There are many examples littering the history of science that tell a story about how a fixation on the facts (and more recently data), and their allegedly virtuous apoliticality, has led us astray.

Consider the story of Geoffrey Marcy. It does not matter, or matters less, that humankind as a whole has made great astronomical discoveries. Instead, it should matter – or matter more – how we go about making them. And Marcy was contemptible because his discoveries were fuelled not just by his appreciation of the facts, so to speak, but also because he pushed women out of astronomy and astrophysics and traumatised them. As a result, consuming the scholarship of Marcy, and Krauss and so many others, feels to me like I am fuelling their transgressions.

Many of these scholars assumed prominence because they drew in grants worth millions to their universities. Their scholarship dealt in facts, sure, but in the capitalist university system, a scholarship also translates to grants and an arbitrarily defined ‘prestige’ that allow universities to excuse the scholars’ behaviour and to sideline victims’ accusations. Some universities even participate in a system derisively called ‘passing the trash’; as BuzzFeed reported in the case of Erik Shapiro in 2017, “the ‘trash’ … refers to high-profile professors who bring status and money to universities that either ignore or are unaware of past scandals.”

So supporting scholars for the virtues of their scholarship alone seems quite disingenuous to me. This is sort of like supporting the use of electric vehicles while ignoring the fact that most of the electricity that powers them is produced in coal-fired power plants. In both cases, the official policy is ultimately geared in favour of maximising profits (more here and here). As such, the enemy here is the capitalist system and our universities’ collective decision to function on its principles, ergo singling scholarship out of for praise seems misguided.

This is also why, though I’ve heard multiple arguments to the contrary, I really don’t know how to separate art from artist, or scholarship from scholar. An acquaintance offered the example of Georges Lemaître, the Belgian Catholic priest and cosmologist who – in the acquaintance’s telling – attempted to understand the world as it was without letting his background as a priest get in the way. I was not convinced, saying the case of Lemaître sounded like a privileged example for its clean distinction between one’s beliefs as a person and one’s beliefs as a scientist. I even expressed suspicion that there might be a reason Lemaître turned to a more mechanistic subject like cosmology and not a more negotiated one like social anthropology.

In fact, Krauss also discovered the world as is in many ways, and those findings do not become wrong for the person he was, or was later found to be. But we must not restrict ourselves to the right–wrong axis, and navigate the good–bad axis as well.

In this time, I also became curious about non-white-male (but including trans-male) scientists who may have written on the same topic – the physics of climate change. So I went googling, finding quite a few results. My go-to response in such situations, concerning the fruits of a poisoned tree, has been to diversify sources – to look for other fruits – because then we also discover new scholarship and art, and empower conventionally disprivileged scholars and artists.

In this regard, the publishers of Krauss’s book also share blame (with Krauss’s universities, which empowered him by failing to create a safe space for students). If publishers are sticking with Krauss instead of, say, commissioning a professor from the Indian Institute of Tropical Meteorology, they are only embellishing preexisting prejudices. They reinforce the notion that they’d much rather redeem an unrepentant white man who has sinned than discover a new writer who deserves the opportunity more. So the publishers are only worsening the problem: they are effectively signalling to all guiltless perpetrators that publishers will help salvage what universities let sink.

At this point, another acquaintance offered a reconciliatory message: that while it’s unwise to dismiss misconduct, it’s also unwise to erase it. So it might be better to let it be but to take from it only the good stuff. Sage words, but therein lay another rub because of a vital difference between the power of fiction versus (what I perceive to be) the innate amorality of scientific scholarship.

Fiction inspires better aspirations and is significantly more redeemable as a result, but I don’t suppose we can take the same position on, say, the second law of thermodynamics or Newton’s third law of motion. Or can we? If you know, please tell me. But until I’m disabused of the notion, I expect it will continue to be hard for me to find a way to rescue the scholarship of a ‘tainted’ scholar from the taint itself, especially when the scholarship has little potential – beyond the implicit fact of its existence, and therefore the ‘freedom of research’ it stands for – to improve the human condition as directly as fiction can.

[Six hours later] I realise I’ve written earlier about remembering Richard Feynman a certain way, as well as Enrico Fermi – the former for misogyny and the latter for a troublingly apolitical engagement with America’s nuclear programme – and that those prescriptions, to remember the bad with the good and to remember the good with the bad, are now at odds with my response to Krauss. This is where it struck me the issue lay: I believe what works for Feynman should work for Krauss as well except in the case of Krauss’s new book.

Feynman was relatively more prolific, since he was also more of a communicator and teacher, than Fermi or Krauss. But while it’s impossible for me to escape the use of Feynman diagrams or Fermi-Dirac statistics if I were a theoretical particle physicist, I still have a choice to buy or boycott the book Surely You’re Joking, Mr Feynman! (1985) with zero consequences for my professional career. If at this point you rebut that “every book teaches us something” so we can still read books without endorsing the authors themselves, I would disagree on the simple point that if you wish to learn, you could seek out other authors, especially those who deserve the opportunity of your readership more.

I expect for the reasons and uncertainty described earlier that the same can go for Krauss and The Physics of Climate Change as well: remember that Krauss was a good physicist and a bad man, and that he was a bad man who produced good physics, but even as other scientists stand on the shoulders of his contributions to quantum physics, I can and will skip The Physics of Climate Change.

Axiomatically, the more we insist that good science communication, an instance of which I believe the book is, is important to inculcate better public appreciation of scientific research, and in the long run improve funding prospects, increase public interest in science-backed solutions to societal problems, draw more students into STEM fields and hold the scientific enterprise accountable in more meaningful as well as efficacious ways, the more science communication itself becomes a stakeholder in the mechanisms that produce scientific work that universities capitalise on, that is currency of this whole enterprise.

Jun 29, 2020
Go easy on the dexamethasone hype
1. The people involved with the RECOVERY clinical trial have announced via statements to the press that they have found very encouraging results about the use of dexamethasone in people with severe COVID-19 who had to receive ventilator support. However, the study’s data isn’t available for independent verification yet. So irrespective of how pumped the trial’s researchers are, wait. Studies in more advanced stages of the publishing process have been sunk before.
2. Dexamethasone is relatively cheap and widely available. But that doesn’t mean it will continue to remain that way in future. The UK government has already announced it has stockpiled 200,000 doses of the drug, and other countries with access to supply may follow suit. Companies that manufacture the drug may also decide to hike prices, foreseeing rising demand, leading to further issues of availability.
3. Researchers found in their clinical trials that the drug reduced mortality among patients with COVID-19 and who needed ventilator support by around 33%, and who needed oxygen by about 20%. This describes a very specific use-case, and governments must ensure that if the drug is repurposed for COVID-19, its use is limited to people who fulfil the specific criteria that benefit from the drug’s use. As the preliminary report notes, “It is important to recognise that we found no evidence of benefit for patients who did not require oxygen and we did not study patients outside the hospital setting.” In addition, dexamethasone is a steroid, and indiscriminate use is quite likely to lead to adverse side effects with zero benefits.
4. The novel coronavirus pandemic is not a tragedy in the number of deaths alone. An important long term effect will be disability, considering the virus has been known to affect multiple parts of the body, including the heart, brain and the kidneys, apart from the lungs themselves, even among patients who have survived. Additionally, it cuts mortality in patients in a later stage of the COVID-19 infection. So go easy on words like ‘game-changer’. Dexamethasone isn’t exactly one because game-changers need to allow people to contract the virus but not fear disability or their lives…
5. … or in fact not fear contracting the virus at all – like a vaccine or an efficacious prophylactic. This is very important, for example, because of what we have already seen in Italy and New York. Many patients who don’t need ventilator support or oxygen care still need hospital care, and the unavailability of hospital beds and skilled personnel can lead to more deaths than may be due to COVID-19. This ‘effect’, so to speak, is more pronounced in developing nations, many of which have panicked and formulated policies that pay way more or way less attention to COVID-19 than is due. In India, for example, nearly 900 people have died due to the lockdown itself.
Jun 17, 2020
The number of deaths averted

What are epidemiological models for? You can use models to inform policy and other decision-making. But you can’t use them to manufacture a number that you can advertise in order to draw praise. That’s what the government’s excuse appears to be vis-à-vis the number of deaths averted by India’s nationwide lockdown.

When the government says 37,000 deaths were averted, how can we know if this figure was right or wrong? A bunch of scientists complained that the model wasn’t transparent, so its output had to be taken with a cupful of salt. But as an article published in The Wire yesterday noted, these scientists were asking the wrong questions – that the number of deaths averted is only a decoy.

So say the model had been completely transparent. I don’t see why we should still care about the number of deaths averted. First, such a model is trying to determine the consequences of an action that was not performed, i.e. the number of people who might have died had the lockdown not been imposed.

This scenario is reminiscent of a trope in many time-travel stories. If you went back in time and caused someone to do Y instead of X, would your reality change or stay the same considering it’s in the consequent future of Y instead of X? Or as Ronald Bailey wrote in Reason, “If people change their behaviour in response to new information unrelated to … anti-contagion policies, this could reduce infection growth rates as well, thus causing the researchers to overstate the effectiveness of anti-contagion policies.”

Second, a model to estimate the number of deaths averted by the lockdown will in effect attempt to isolate a vanishingly narrow strip of the lockdown’s consequences to cheer about. This would be nothing but extreme cherry-picking.

A lockdown has many effects, including movement restrictions, stay-at-home orders, disrupted supply of essential goods, closing of businesses, etc. Most, if not all, of them are bound to exact a toll on one’s health. So the number of deaths the lockdown averted should be ‘adjusted’ against, say, the number of people who couldn’t get life-saving surgeries, the number of migrant labourers who died of heat exhaustion, the number of TB patients who developed MDR-TB because they couldn’t get their medicines on time, even the number of daily-wage earners’ children who died of hunger because their parents had no income.

So the only models that can hope to estimate a meaningful number of deaths averted by the lockdown will also have simplified the context so much that the mathematical form of the lockdown will be shorn of all practical applicability or relevance – a quantitative catch-22.

Third, the virtue of the number of deaths averted is a foregone conclusion. That is, whatever its value is, it can only be a good thing. So as an indisputable – and therefore unfalsifiable – entity, there is nothing to be gained or lost by interrogating it, except perhaps to elicit a clearer view of the model’s innards (if possible, and only relative to the outputs of other models).

Finally, the lockdown will by design avert some deaths – i.e. D > 0 – but D being greater than zero wouldn’t mean the lockdown was a success as much D‘s value, whatever it is, being a self-fulfilling prophecy. And since no one knows what the value of D is or what it ought to be, even less what it could have been, a model can at best come up with a way to estimate D – but not claim a victory of any kind.

So it would seem the ‘number of deaths averted’ metric is a ploy disguised as a legitimate mathematical problem whose real purpose is to lure the ‘quants’ towards something they think challenges their abilities without realising they’re also being lured away from the more important question they should be asking: why solve this problem at all?

Jun 10, 2020
The costs of correction

I was slightly disappointed to read a report in the New York Times this morning. Entitled ‘Two Huge COVID-19 Studies Are Retracted After Scientists Sound Alarms’, it discussed the implications of two large studies of COVID-19 recently being retracted by two leading medical journals they were published in, the New England Journal of Medicine and The Lancet. My sentiment stemmed from the following paragraph and some after:

I don’t know if just these two retractions raise troubling questions as if these questions weren’t already being asked well before these incidents. The suggestion that the lack of peer-review, or any form of peer-review at all in its current form (opaque, unpaid) could be to blame is more frustrating, as is the article’s own focus on the quality of the databases used in the two studies instead of the overarching issue. Perhaps this is yet another manifestation of the NYT’s crisis under Trump?

One of the benefits of the preprint publishing system is that peer-review is substituted with ‘open review’. And one of the purposes of preprints is that the authors of a study can collect feedback and suggestions before publishing in a peer-reviewed journal instead of accruing a significant correction cost post-publication, in the form of corrections or retractions, both of which continue to carry a considerable amount of stigma. So as such, the preprints mode ensures a more complete, a more thoroughly reviewed manuscript enters the peer-review system instead of vesting the entire burden of fact-checking and reviewing a paper on a small group of experts whose names and suggestions most journals don’t reveal, and who are generally unpaid for their time and efforts.

Shambles. @TheLancet and @NEJM should immediately submit themselves to an independent inquiry concerning the standard of their editing and internal review processes. https://t.co/TyYTbgeyQn
— Chris Chambers (@chrisdc77) June 4, 2020

In turn, the state of scientific research is fine. It would simply be even better if we reduced the costs associated with correcting the scientific record instead of heaping more penalties on that one moment, as the conventional system of publishing does. ‘Conventional – which in this sphere seems to be another word for ‘closed-off’ – journals also have an incentive to refuse to publish corrections or perform retractions because they’ve built themselves up on claims of being discerning, thorough and reliable. So retractions are a black mark on their record. Elisabeth Bik has often noted how long journals take to even acknowledge entirely legitimate complaints about papers they’ve published, presumably for this reason.

There really shouldn’t be any debate on which system is better – but sadly there is.

Jun 6, 2020
The life and death of ‘Chemical Nova’

You know how people pretend to win an Oscar or a Nobel Prize, right? Many years ago, I used to pretend to be the author of a fictitious but, blissfully unmindful of its fictitiousness, award-winning series of articles entitled Chemical Nova. In this series, I would pretend that each article discussed a particular point of intersection between science and culture.

The earliest idea I had along these lines concerned soap. I would daydream about how I was celebrated for kickstarting a social movement that prized access to soap and ability to wash one’s hands under running water, and with this simple activity beat back the strange practice among many of refusing to wash one’s toilet oneself, instead delegating the apparently execrable task to a housemaid.

The fantastic value of Chemical Nova should be obvious: it represented, at least to me, the triumph of logic and reasoning above class-commitments and superstition. The fantasy took shape out of my longstanding ambition to beat down a stubborn Creature, for many years shapeless, that often caused a good review, essay or news report to inspire only cynicism, derision and eventually dismissal on the part of many readers. It was quickly apparent that the Creature couldn’t be subdued with deductive reasoning alone, but for which one had to take recourse through politics and individual aspirations as well, no matter how disconnected from the pretentious ‘quest for truth’ these matters were.

Chemical Nova dissipated for a few years as I set about becoming a professional journalist – until I had occasion to remember it after Narendra Modi’s election as prime minister in 2014. And quickly enough, it seemed laughable to me that I had assumed upper-caste people wouldn’t know how soap worked, or at least of its cleansing properties. An upper-caste individual invested in the continuation of manual scavenging would simply feel less guilty with a bar of soap placed in his dirty bathroom: for scavengers to wash their hands and not be at risk of contracting any diseases.

The belief that ‘the job is theirs to perform’ could then persist unfettered, rooted as it was in some sort of imagined befoulment of the soul – something one couldn’t cleanse, out of reach of every chemical reagent, or even affect in any way except through a lifetime of suffering.

It was a disappointing thought, but in my mind, there was still some hope for Chemical Nova. Its path was no longer straightforward at all insofar as it had to first make the case that the mind, the body and the community are all that matter, that that’s how one’s soul really takes shape, but its message – “ultimately, wash your hands” – still was an easy one to get across. I was tempted and I continued to wait.

However, earlier today, the Creature bared itself fully, exposing not itself as much as the futility of ideas like Chemical Nova. An advertisement appeared in a newspaper displaying a pair of hands kneading some dough, with the following caption: “Are you allowing your maid to knead atta dough by hand? Her hands may be infected.” The asset encouraged readers of the newspaper to buy Kent’s “atta maker & bread maker” instead, accompanied by a photograph of Hema Malini smiling in approval.

Malini has been the brand ambassador for Kent since 2007 and the incumbent Lok Sabha MP from Mathura since 2014. I’m not sure of the extent to which she knew of the advertisement’s contents before her face (and her daughter’s) appeared on it. Her affiliation since 2004 with the Bharatiya Janata Party (BJP), known for its favouritism towards upper-caste Hindus (to put it mildly), doesn’t inspire confidence but at the same time, it’s quite possible that Malini’s contract with Kent allows the company to include her face in promotional materials for a predefined set of products without requiring prior approval in each instance.

But even if Malini had never been associated with the product or the brand, Chemical Nova would have taken a hit because I had never imagined that the Creature could one day be everywhere at once. The chairman of Kent has since apologised for the advertisement, calling it “unintentional” and “wrongly communicated”. But it seems to me that Kent and the ad agency it hired continue to err because they don’t see the real problem: that they wrote those words down and didn’t immediately cringe, that those words were okayed by many pairs of eyes before they were printed.

The triumph of reason and the immutability of chemical reagents are pointless. The normalisation of exclusion, of creating an ‘other’ who embodies everything the in-group finds undesirable, is not new – but it has for the most part been driven by a top-down impulse that often originates in the offices of Narendra Modi, Amit Shah or some senior BJP minister, and often to distract from some governmental failure. But in the coronavirus pandemic, the act of ‘othering’ seems to have reached community transmission just as fast as the virus may have, finding widespread expression without any ostensible prompt.

And while Kent has been caught out evidently because it was the ‘loudest’, I wonder how many others don’t immediately see that what they are writing, saying, hearing or reading is wrong, and let it pass. As Arundhati Roy wrote earlier this week, the attainment of ‘touchlessness’ seems to be the new normal: in the form of a social condition in which physical distance becomes an excuse to revive and re-normalise untouchabilities that have become taboo – in much the same way soap became subsumed by the enterprise it should have toppled.

Examples already abound, with ministers and corporate uncles alike touting the prescient wisdom of our Hindu ancestors to greet others with a namaste instead of shaking hands; to maintain aachaaram, a collection of gendered practices many of which require the (Brahmin) practitioner to cleanse themselves of ‘spiritual dirt’ through habits and rituals easily incorporated into daily life; and now, to use machines that promise to render, in Roy’s words, “the very bodies of one class … as a biohazard to another”.

It started with a bang, but Chemical Nova slips quietly into the drain, and out of sight, for it is no match for its foe – the Creature called wilful ignorance.

Featured image: A snapshot of William Blake’s ‘The Great Red Dragon and the Woman Clothed with the Sun’, c. 1805-1810.

May 27, 2020
A caveat for peer-review

Now that more researchers are finding more holes in the study in The Lancet, which claimed hydroxychloroquine – far from being a saviour of people with COVID-19 – actually harms them, I wonder where the people are who’ve been hollering that preprint servers be shut down because they harm people during a pandemic.

The Lancet study, entitled ‘Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis’, was published online on May 22, 2020. Quoting from the section on its findings:

After controlling for multiple confounding factors (age, sex, race or ethnicity, body-mass index, underlying cardiovascular disease and its risk factors, diabetes, underlying lung disease, smoking, immunosuppressed condition, and baseline disease severity), when compared with mortality in the control group, hydroxychloroquine, hydroxychloroquine with a macrolide, chloroquine, and chloroquine with a macrolide were each independently associated with an increased risk of in-hospital mortality. Compared with the control group, hydroxychloroquine, hydroxychloroquine with a macrolide, chloroquine, and chloroquine with a macrolide were independently associated with an increased risk of de-novo ventricular arrhythmia during hospitalisation.

I assume it was peer-reviewed. According to the journal’s website, there is an option for researchers to have their submission fast-tracked if they’re so eligible (emphasis added):

All randomised controlled trials are eligible for Swift+, our fastest route to publication. Our editors will provide a decision within 10 working days; if sent for review this will include full peer review. If accepted, publication online will occur within another 10 working days (10+10). For research papers, which will usually be randomised controlled trials, judged eligible for consideration by the journal’s staff will be peer-reviewed within 72 h and, if accepted, published within 4 weeks of receipt.

The statistician Andrew Gelman featured two critiques on his blog, both by a James Watson, on May 24 and May 25. There are many others, including from other researchers, but these two provide a good indication of the extent to and ways in which the results could be wrong. On May 24:

… seeing such huge effects really suggests that some very big confounders have not been properly adjusted for. What’s interesting is that the New England Journal of Medicine published a very similar study a few weeks ago where they saw no effect on mortality. Guess what, they had much more detailed data on patient severity. One thing that the authors of the Lancet paper didn’t do, which they could have done: If HCQ/CQ is killing people, you would expect a dose (mg/kg) effect. There is very large variation in the doses that the hospitals are giving … . Our group has already shown that in chloroquine self-poisoning, death is highly predictable from dose. No dose effect would suggest it’s mostly confounding. In short, it’s a pretty poor dataset and the results, if interpreted literally, could massively damage ongoing randomised trials of HCQ/CQ.

On May 25:

The study only has four authors, which is weird for a global study in 96,000 patients (and no acknowledgements at the end of the paper). Studies like this in medicine usually would have 50-100 authors (often in some kind of collaborative group). The data come from the “Surgical Outcomes Collaborative”, which is in fact a company. The CEO (Sapan Desai) is the second author. One of the comments on the blog post is “I was surprised to see that the data have not been analysed using a hierarchical model”. But not only do they not use hierarchical modelling and they do not appear to be adjusting by hospital/country, they also give almost no information about the different hospitals: which countries (just continent level), how the treated vs not treated are distributed across hospitals, etc.

(Gelman notes in a postscript that “we know from experience that The Lancet can make mistakes. Peer review is nothing at all compared to open review.” – So I’m confident the study’s paper was peer-reviewed before it was published.)

Perhaps it’s time we attached a caveat to claims drawn from peer-reviewed papers: that “the results have been peer-reviewed but that doesn’t have to mean they’re right”, just as journalists are already expected to note that “preprint papers haven’t been peer-reviewed yet”.

May 27, 2020
The WHA coronavirus resolution is not great for science

On May 19, member states of the WHO moved a vote in the World Health Assembly (WHA), asking for an independent investigation into the sources of the novel coronavirus.

Their exact demands were spelled out in a draft resolution that asked the WHO to, among other things, “identify the zoonotic source of the virus and the route of introduction to the human population, including the possible role of intermediate hosts, including through efforts such as scientific and collaborative field missions”.

The resolution was backed by 62 countries, including India, and the decision to adopt it was passed with 116 votes in favour, out of 194. This fraction essentially indicates that the overwhelming majority of WHO’s member states want to ‘reform’ the organisation towards a better response to the pandemic, especially in terms of obtaining information that they believe China has been reluctant to share.

The resolution follows from Australia’s demand in April 2020 for a public inquiry against China, suggesting that the Asian superpower was responsible for the virus and the global outbreak (not surprisingly, US President Donald Trump expressed his support). Together with the fact that the document doesn’t once mention China, the resolution is likely an expression of concern that seeks to improve international access to biological samples, specific locations and research data necessary to find out how the novel coronavirus came to infect humans, and which animal or avian species were intermediate hosts.

As it happens, this arguably legitimate demand doesn’t preclude the possibility that the resolution is motivated, at least in part, by the need to explore what is in many political leaders’ view the ‘alternative’ that the virus originated in a Chinese lab.

The WHA vote passed and the independent investigation will happen – but by who or how is unclear. Let’s assume for now that some team or other comes together and conducts the requisite studies.

What if the team does find that the virus is not lab-made? Will those WHO member states, and/or their politicians back home, that were in favour of the resolution to explore the ‘lab hypothesis’ let the matter rest? Or will they point fingers at the WHO and claim it is too favourable to China, as President Trump has already done and to which the resolution’s reformatory language alludes?

In fact, the investigation is unlikely to zero in on the virus’s origins if they were natural because too much time has passed since the first zoonotic spillover event. The bread crumbs could have long faded by the time the investigation team sets out on its task. It won’t be impossible, mind, but it will be very difficult and likely require many months to conclude.

But what if the investigation somehow finds that the virus was engineered in a lab and then leaked, either deliberately or accidentally? Will the scientists and those who believed them (including myself) stand corrected?

They will not. There’s a simple reason why: they – we – have thus far not been given enough evidence to reach this conclusion.

Indeed, there is already sufficient explanation these days to claim that the novel coronavirus is of natural origin and insufficient explanation that it was engineered. A study published on March 17, 2020, collected evidence for the former (and many others continue to do so). An excerpt from the conclusion:

The genomic features described here may explain in part the infectiousness and transmissibility of SARS-CoV-2 in humans. Although the evidence shows that SARS-CoV-2 is not a purposefully manipulated virus, it is currently impossible to prove or disprove the other theories of its origin described here. However, since we observed all notable SARS-CoV-2 features, including the optimised RBD and polybasic cleavage site, in related coronaviruses in nature, we do not believe that any type of laboratory-based scenario is plausible.

If there is any animosity at all directed at China for supposedly engineering the virus, the countries that backed the resolution could only have done so by actively ignoring the evidence that already exists to the contrary.

In this particular case, it becomes extremely important for the representatives of these countries to explain why they think the evidence that scientists have not been able to find actually exists, and that they are simply yet to discover it. That is, why do they think some pieces are missing from the puzzle?

There is of course room for a deeper counter-argument here, but it isn’t entirely tenable either. One could still argue that there might be a larger ‘super-theory’ that encompasses the present one even as it elucidates a non-natural origin for the virus. This is akin to the principle of correspondence in the philosophy of science. The advent of the theories of relativity did not invalidate the Newtonian theory of gravity. Instead, the former resemble the latter in the specific domain in which the latter is applicable. Similarly, a ‘super-theory’ of the virus’s origins could point to evidence of bioengineering even as its conclusions resemble the evidence I’m pointing to to ascertain that the virus is natural.

But even then, the question remains: Why do you think such a theory exists?

Without this information, we are at risk of wasting our time in each pandemic looking for alternate causes that may or may not exist, many of which are quite politically convenient as well.

Perhaps we can assimilate a sign of things to come based on Harsh Vardhan’s performance as the chairman of the WHA’s executive board. Vardhan was elected into this position at the same WHA that adopted the draft resolution, and his highest priority is likely to be the independent investigation that the resolution calls for. As it happens, according to OP8 of the resolution, the resolution:

… calls on international organisations and other relevant stakeholders to … address, and where relevant in coordination with Member States, the proliferation of disinformation and misinformation particularly in the digital sphere, as well as the proliferation of malicious cyber-activities that undermine the public health response, and support the timely provision of clear, objective and science-based data and information to the public.

India as a member state is certainly a stakeholder, and Nitin Gadkari, one of the country’s senior ministers, recently said in an interview that the novel coronavirus was made in a lab. This is misinformation plain and simple, and goes against the call for the “timely provision of clear, objective and science-based information to the public”. Will the chair address this, please – or even future instances of such imprudence?

Ultimately, unless the investigation ends with the conspiracists changing their minds, the only outcome that seems to be guaranteed is that scientists will know their leaders no longer trust their work.

Featured image: The assembly hall of the Palace of Nations, Geneva, where the World Health Assembly usually meets. Photo: Tom Page/Wikimedia Commons, CC BY-SA 2.0.

May 22, 2020
The Wolfram singularity

I got to this article about Stephen Wolfram’s most recent attempt to “revolutionise” fundamental physics quite late, and sorry for it because I had no idea Wolfram was the kind of guy who could be a windbag. I haven’t ever had cause to interact with his work or his software company (which produced Wolfram Mathematica and Wolfram Alpha), so I didn’t know really know much about him to begin with. But I expected him, for reasons I can’t explain, to be more modest than he comes across as in the article.

The article was prompted in the first place by a preprint paper Wolfram and a colleague published earlier this year in which they claimed they had plotted a route to a fundamental theory of everything. Physics currently explains the universe with a combination of multiple theories that don’t really fit together. A ‘theory of everything’ is the colloquial name of a universal theory that many physicists argue exist and which could explain everything about the universe in a self-consistent manner.

Wolfram’s preprint paper was startling as things go not because of its substance but because a) he made no attempts to engage with the wider community of physicists that has been working on the same problem for decades, and b) for Wolfram’s insistence that those dismissing its conclusions are simply out to dismiss him. Consider the following portions:

“I do fault myself for not having done this 20 years ago,” the physicist turned software entrepreneur says. “To be fair, I also fault some people in the physics community for trying to prevent it happening 20 years ago. They were successful” [emphasis added].
“The experimental predictions of [quantum physics and general relativity] have been confirmed to many decimal places—in some cases, to a precision of one part in [10 billion],” says Daniel Harlow, a physicist at the Massachusetts Institute of Technology. “So far I see no indication that this could be done using the simple kinds of [computational rules] advocated by Wolfram. The successes he claims are, at best, qualitative.” …
“Certainly there’s no reason that Wolfram and his colleagues should be able to bypass formal peer review,” Katie Mack says. “And they definitely have a much better chance of getting useful feedback from the physics community if they publish their results in a format we actually have the tools to deal with.”

Reading of this attitude brought to mind an episode from six or seven weeks ago, after a pair of physicists had published a preprint paper modelling the evolution of the COVID-19 epidemic in India and predicting that multiple lockdowns instead of just one would work better. The paper was one of many that began to show up around that time, each set of authors fiddling with different parameters according to their sense of the world to reach markedly different conclusions (a bit of ambulance-chasing if you ask me).

The one by the two physicists was singled out for bristling criticism by other physicists because – quite like the complaints against Wolfram – their paper allegedly described a model that seemed to be able to reach any conclusion if you tweaked its parameters enough, and because the duo hadn’t clarified this and other important caveats in their interviews to journalists.

Aside 1 – In physics at least, it’s important for theories to be provable in some domains and falsifiable in others; if a theory of the world is non-falsifiable, it’s not considered legitimate. In Wolfgang Pauli’s famous words, it becomes ‘not even wrong’.

Aside 2 – Incidentally, Harlow – quoted above from the article – was one of the physicists defending physicists’ freedom to model what they will but agreed with the objection that they also need to be honest with journalists about their assumptions and caveats.

In a lengthy Facebook discussion that followed this brouhaha, someone referred to a Reddit post created three days earlier in which a physicist appealed to his peers to stop trying to model the pandemic – in his words, to “cut that shit out” – because a) no physicist could hope to do a better job than any other trained epidemiologist, and b) every model a physicist attempted could actually harm lives if it wasn’t done right (and there was a good chance it was at least incomplete).

Wolfram is guilty of the same thing: his preprint paper won’t harm lives, but the mortal threat is the only thing missing from his story; it’s otherwise rife with the same problems. His hubristic remark in the article’s denouement – that he deserves “better” questions than the ones other physicists were asking him in response to his “revolutionary” paper – indicates Wolfram thinks he’s done a great job but it’s impossible to see people like him as anything more than windbags convinced of their intellectual superiority and ability to singlehandedly wrestle hideously intractable problems to the ground. I, and likely other editors as well, have glimpsed this attitude on the part of some authors who dismiss criticism of their pieces as criticism of anything but their unclear writing, and some others who refuse to be disabused of a conviction that their conclusion is particularly fascinating.

I’d like to ask Wolfram what I’d like to ask these people as well: What have you hit on that you think others haven’t in all this time, and why do you think all of them missed it? Granted, everyone is allowed their ‘eureka’ moment, but anyone who claims it on the condition that he not be criticised is not likely to be taken seriously. More importantly, he may not even deserve to be taken seriously if only because, to adapt Mack’s line of reasoning, he undermines the very thing on which modern science is founded, the science he claims to be improving: processes, not outcomes; involving communities, not individuals.

Featured image credit: Anna Shvets/Pexels.

May 20, 2020