Welcome to the Science Fictions newsletter. Or, if you’re one of my previous subscribers from Substack – welcome to the newsletter’s new home. Science Fictions is continuing here at i, and will now come out every Thursday. I’ll keep covering controversies in science, picking apart bad research papers, and digging into the data on interesting scientific questions.
As ever, I’d love for you to get in touch with any stories of bad science, or anything you’d like me to look into – my contact details are at the bottom of this page.
“I’ll NEVER give up my gas stove. If the maniacs in the White House come for my stove, they can pry it from my cold dead hands. COME AND TAKE IT!!”
Those were the immortal words of the ex-US Surgeon General, Dr. Ronny Jackson, in a tweet last Tuesday. He was reacting to a statement from an official from the Consumer Product Safety Commission, who said he was considering advocating for a ban on gas stoves for health and safety reasons. The official was himself reacting to a new study that claimed that a large number of US asthma cases in children could be traced back to gas-burning kitchen cookers.
The statement about a potential ban has since been walked back – and even President Biden’s spokesperson had to say that the President (who loves to grill) isn’t in favour of banning gas stoves. But it’s ignited (no pun intended) a major debate on the health effects of indoor air pollution – and has settled itself nicely on to the pre-existing US culture war, with left-leaning Democrats seemingly open to the idea that gas stoves might be harmful, and right-leaning Republicans – well, you saw Jackson’s tweet above.
There are very many health complaints that have been studied in relation to indoor air quality, and in some cases specifically to gas stoves in the home. Cancer is one, and cognitive dysfunction is another (the latter was noted by Democrat Congresswoman Alexandria Ocasio-Cortez in a snarky response to Jackson).
But since childhood asthma has been the main topic of recent discussion, that’s what we’ll focus on here. How much research has been done on this? Is any of that research any good? Let’s read all of it to find out.
Wheezy does it
The funny thing about the current flare-up (no pun intended, again) about gas stoves and asthma is that no new evidence has been produced. The December 2022 study that’s got people so exercised is just a brief note that does a little bit of rejigging of some old statistics.
Before we get to that, let’s look at the basic logic of the claim. First, there’s no debate that gas stoves produce air pollution. Certainly they’re a lot less polluting than an open fire – still used for cooking in many countries and a source of even bigger health worries – but if you sit an air-quality meter next to your cooker next time you’re frying an egg, you’ll see that it picks up a spike in nitrogen dioxide (NO2), as well as other potential air pollutants such as carbon monoxide (CO) and fine particulate matter (PM2.5).
But of course, the dose makes the poison: just because cooking increases the quantities of these substances in the air doesn’t mean that it produces them to dangerous levels. We all know that a large CO leak in your house can be lethal (it’s why the UK mandates landlords to install CO monitors as well as smoke alarms in their properties). But it’s less clear that the smaller amount of CO (or, for that matter, NO2 and PM2.5, which are both known to be toxic at high levels) released by a kitchen cooker would affect human health.
When we’re talking about asthma, the main focus is on NO2. The theory—supported by some direct experiments—is that, for unclear chemical reasons, inhaling NO2 “primes” the specific type of white blood cells that react to allergens and cause the lung tissue inflammation that’s a characteristic of asthma. Primed in this way, the cells have a greater response to the next allergen that enters the system, and your asthma symptoms will be worse. And indeed, there’s at least some evidence that asthmatics who are exposed to NO2 have a small drop in the amount of air they can exhale immediately afterwards.
These two findings – gas stoves produce NO2; NO2 seems to exacerbate asthma in direct lab experiments – spurred epidemiologists on to study the real-world correlation between how much gas stove use there is in your house, and two different outcomes: whether you get asthma in the first place, and how bad your asthma is.
It was the first of those questions – that of whether gas stoves were linked to children getting asthma – that was the focus of the new study. Except, as I noted above, it’s not much of a new study. Their plan was to: (a) collect all the evidence that’s already been published on gas stoves and asthma so far, and (b) use it to calculate how many current asthma cases are due to the use of gas stoves. So, again, it’s not new data – just a new interpretation.
To do part (a), they started with a meta-analysis of gas stoves and asthma that was published in 2013, which put together all the relevant studies up to that point. They then looked at the scientific literature to see if there were any relevant studies published since then that they could add. And… there weren’t any. Not one study fit the bill (we’ll see below that there have been more recent studies, but they did something a little different from this one, so didn’t make it into this analysis).
So, to do (b), all they did was take the estimates from the meta-analysis and use them to calculate a “Population Attributable Fraction” (PAF) – the proportion of asthma cases that are, given all the evidence, attributable to gas cooking (another way of putting it would be to say, “how many asthma cases would be prevented if we all stopped using gas stoves tomorrow?”).
They plugged in the data from the meta-analysis, and from a US Household Survey that collected data on gas stove use from nine US states. And out came the numbers, ranging from 3 per cent in Florida to 21.1 per cent in Illinois: on average, they found that 12.7 per cent of asthma cases in the US as a whole were attributable to gas stove use.
Straightforward, right? Well… not really.
Up in flames
The first problem is that the word “attributable” in “population attributable fraction” is misleading. It’s a bit like when researchers say that X “explained the variance” in Y: the language implies that one thing is causing another, but the numbers don’t really. You can, as these researchers have, calculate the PAF with an underlying assumption that one thing causes another – that stopping gas stove use would mean less asthma – but that’s a big assumption, given that all the studies included in the meta-analysis were correlational. That is, they just collected data on the extent to which gas stoves were used and the rates of asthma, without doing any kind of experiment or intervention to check what causes what.
And the causality isn’t certain. Imagine a world where people who tend to have gas stoves (as opposed to electric or induction hobs) also tend to be poorer. And where poorer people live in worse parts of town with lower-quality, damp housing and more pollution from traffic on the road outside. It’s not actually that hard to imagine – but it would mean that, even though it would look as if there was a correlation between gas cookers and asthma in the study, the true cause of the asthma could be any one of many different factors. So much for “attributable”; remove the gas stoves and you might not in fact prevent 12.7 per cent of anything.
But it’s actually worse than that. If you look in detail at the meta-analysis, you find that what’s being measured across the 19 different studies they collected is very vague. The specific variable they say they measured was high vs. low gas cooker use. But each study was included “independently of the exact definition of high and low exposure” – and might have measured it in very different ways. There’s nothing on how much the gas stove was used, how often the child was in the room; there’s no direct comparison between different kinds of cookers. So one of the sides of our equation—the one with the actual “exposure” that we’re interested in—is rather mushy and unclear.
Far clearer are the studies that were able to measure NO2 in the home and correlate it with levels of asthma. In those ones we have a quantifiable variable to study: for an increase in this specific amount of NO2 (in this case 15 parts per billion) we see this increased risk of asthma (in this case a 9 per cent increased risk). But there were only five such studies, and none of them were included in this new PAF research anyway.
Add that to the fact that the PAF calculation was made on the basis of data from just nine US states (and it’s not even clear what those data were, or how they related to the high vs. low gas cooking exposure in the meta-analysis; it seems suspicious that there were such wide differences in gas cooker use between relatively-similar US states), and it becomes my contention that the 12.7 per cent number is basically meaningless. It’s too far abstracted from the measurements and data to be useful.
So should this send us all scuttling back to our gas cookers to fry up an enormous Full English? Do we agree with the American Gas Association, who put out a statement last week making the point that the new study was only correlational, and accusing its authors of “advancing bad science” and “recklessly creating fear”?
Not quite yet. Even if you disregard this new study—and you probably should!—there are a whole bunch of other studies we need to look at before we decide whether or not to worry about gas cookers and asthma.
Now we’re cooking with gas
Remember I said above that some of the studies looked not at whether people got asthma to begin with, but how bad their symptoms were? Here’s where I found a much more convincing study. The same year the meta-analysis came out, 2013, a year-long study measured levels of NO2 in the home and the severity of children’s asthma and wheeze symptoms.
What’s compelling about this study was that it found dose-dependent results: that is, it wasn’t just that there was an overall relation between NO2 and asthma, but for every increase in NO2 in the household, there was a related increase in asthma symptoms. Dose-dependency is one of the classic “Bradford Hill criteria” that epidemiologists use for assessing causality: it’s not 100 per cent knockdown evidence, but it’s certainly a tick on the side of “NO2 in the home causes asthma symptoms”. And the study directly answered our question from above: these weren’t abnormally high levels of NO2.
Now, in these kinds of articles, this would usually be the point where I’d say “of course, all these studies are just correlational – but you can’t exactly do a randomised controlled trial on this topic, can you?! You can’t randomly have kids huffing nitrogen dioxide to see how it affects their asthma!”. Good joke. Everybody laugh. Roll on snare drum. Curtains.
But that would be to display a lack of imagination. In fact, people have done randomised controlled trials in this sphere. It’s just that—rather obviously, in retrospect—they don’t randomise how much NO2 is put in to the air, but instead, how much is taken out.
In other words, they’re trials of air purifiers, or other interventions aimed at removing or scrubbing toxins from the air. The first one was from 2004, where eight randomly-selected primary schools in Australia had their old gas heating systems replaced with newer “flued” gas systems, or electric ones, both of which produce less NO2. A further ten schools kept their old gas boilers. The resulting study found that breathing difficulties and asthma attacks were substantially lower in the schools with the changed heating systems, though they only looked at 199 individual children in total (I have a slight concern about this study because one of the authors seems to have published it a few years earlier as their PhD thesis, and the results there look rather different).
In 2008 a study in New Zealand randomised the installation of newer, less polluting home heaters in the households of 409 asthmatic children. After a few months, children living in the homes with new heaters had lower levels of asthma symptoms and better overall health (and a 2011 follow-up confirmed that this was related to NO2 levels).
You’ll have noticed that neither of these trials were specifically about gas stoves. One was in a school, where stoves are presumably not a major pollutant, and one was specifically about heaters. We had to wait until 2022 for a randomised controlled trial to address the question we really want answered. In that trial, they recruited 126 US kids living in homes that definitely had gas stoves, and randomised them to either get an NO2-filtering device, or a sham device that didn’t actually filter anything (this latter aspect, the placebo condition, helped them rule out the possibility that participants who knew they had an air filter would, deliberately or otherwise, under-report their asthma symptoms).
They did find that the children with the genuine air purifiers had fewer asthma symptoms after five weeks. But the difference was too small to be statistically significant – it was consistent with random chance fluctuations. In any case, it’s likely that the study was too small to provide any useful information.
I looked in vain for any econometric-style studies that used a natural experiment – say, a sudden rise or drop in gas prices in some country that might’ve caused people to change how much they used their gas stove. Such a study could look for differences in rates of asthma diagnoses for the affected households, and would be able to do so on a much larger scale than the randomised trials we’ve just seen, But as far as I could see, nothing like this exists (economists – if you’re reading this, please go out and do that study!).
Say NO2 to gas stoves?
What are we to make of all this? I think we can add up everything we’ve learned and draw a tentative conclusion. It’s plausible that air pollution like NO2—especially indoors, in unventilated rooms where it can hang around—is harmful to the lungs, since we have biomedical studies that point in that direction. The correlational studies, taken as a whole, do point in the direction of gas stoves being damaging (and, as noted right at the top, asthma is far from the only bad health outcome that’s correlated with indoor air pollution). The dose-response study is definitely consistent with the idea that worse air pollution leads to worse asthma. And there are so few randomised trials that they’re at this point a bit of a distraction, not to be taken too seriously as evidence.
Put all this together and there is a credible case that gas stove use in the home at least exacerbates asthma symptoms. I don’t think at this point we need to rip out our gas cookers. Ronny Jackson needn’t worry that we’ll be prying anything from anyone’s dead hands. But since it’s a very low-cost, low-difficulty intervention, it probably makes sense to open a window while (or just after) using a gas cooker, or to use a cooker hood with a good extractor fan, especially if you have children nearby.
Some people find it surprising, or even ridiculous, that health officials could suddenly be complaining about such a normal, everyday household appliance. Surely it’s nanny-state overreach! Surely it’s being done for some kind of ulterior, political motive! Maybe, in some cases, there’s something to that.
But just remember: for decades, we used to routinely make pipes, paint, and petrol using lead. It now seems totally obvious that we had to remove this poison from our everyday lives – a task that’s now largely done. It’s very possible—though only after a lot more research – that the humble gas stove will go exactly the same way.
Science link of the week
There was one great piece of news this week: the dramatic decline in deaths from cancer in the US. New data shows that, across the last 20 years, death rates ticked down by 27% (slightly more in males, 30%, than in females, 25%).
That’s largely due to lower levels of smoking, but also better cancer care, which is of course driven by scientific advances such as vaccines that indirectly prevent cancer (by immunising people to, for example, the human papillomavirus).
It’s the sort of thing that should make you feel optimistic about the future. As my friend Saloni Dattani from Our World in Data puts it: “People tend to see cancer as an inevitable part of ageing. What if it’s not?”
This is Science Fictions with Stuart Ritchie, a subscriber-only newsletter from i. If you’d like to get this direct to your inbox, every single week, you can sign up here.
You can also get in touch with any stories of bad science by email at [email protected].