In the last week there have been some alarming reports about the effect magnetic fields can have on unborn babies. Apparently ‘power lines’ and ‘electronic gadgets’ and ‘pylons [are] linked to babies’ asthma’. Really? I mean, really?
Behind the scary headlines is a scary paper. It wasn’t misrepresented, or blown out of proportion. That’s really what it said: ‘high maternal MF [magnetic field] levels in pregnancy may increase the risk of asthma in offspring’. And the lead author himself is quoted as saying that ‘pregnant women should try to limit their exposure to known MF sources’. So what’s his evidence, I hear you ask? Good question, you. Walk with me.
The latest in maternity wear.
De-Kun Li and his pals recruited 1,063 public-spirited pregnant ladies in the San Francisco area and strapped a measuring device to each of them for 24 hours. These meters took a reading of the magnetic field every 10 seconds and the researchers used the median reading for the day as a gauge of ‘overall MF exposure during pregnancy’. Eventually the authors tallied up the medical records of 626 resultant babies. 130 of them developed asthma (21%). If you look separately at the top tenth of all the exposure readings, 33% of those kids developed asthma (21 of 63). If you look at the bottom tenth, 14% did (11 of 81). And apparently if you look at everyone across the spectrum and do some maths, the kids’ risk of asthma increased by 15% for every 1 milligauss (mG) increase in their mums’ magnetic field exposure (a typical reading was ~1.0 mG and 80% of readings fell between 0.3 and 2.0 mG). So there is a relationship, and it’s there even when the researchers do more maths to ward off any possible influence of mums with high exposure also smoking or having asthma themselves.
The 'right hand rule' of magnetic fields and current. NB: actually grabbing hold of a current-carrying wire carries health consequences and is not recommended.
It might help to talk briefly about what these gadgets actually measured. They were set up to record electromagnetic (EM) fields generated by mains power wires and devices: specifically, fields alternating (flipping direction) at a rate of 40-800Hz. If we dive back for a moment into high-school physics (if it were any more sophisticated you would need a better teacher than I) you may remember that any wire carrying a current generates a perpendicular magnetic field, circling around the wire in a specific direction. If that current reverses, as in mains power which alternates 50 times each second (50 Hz), then the direction of the magnetic field also switches. There’s an associated electric field too – hence ‘EM’ field; they’re fairly inextricably linked – and really it’s the electric field, if anything, that might worry us. Electric fields have more obvious interactions with living cells, but don’t penetrate as far and are easily blocked by obstructions like walls and doors; magnetic fields need metal shielding to block them but have only very minor biological effects. Both types of field, as generated by mains power, are extremely weak. While these recordings in expectant San Francisco homes hover around 1 mG, people there and all over our magnetised planet live and thrive in a non-alternating, pretty constant, pretty inescapable magnetic field of about 500 mG.
It is difficult to say how well these measurements reflect the pregnant mums’ overall exposure to EM fields. The scientists asked them if the recording day was “typical” but the 226 mums who answered no were still included. A single day of numbers, from which the median is plucked as an ambassador, says nothing about just how much their behaviour (and exposure) varied throughout pregnancy. We’re not even told whether the recordings were made early or late or all over the place during the 626 relevant pregnancies.
What worries me the most, though, is that all those long-suffering mums were actually burdened with EM field meters for another purpose entirely. The recordings were made back in 1996-98 for a study of miscarriage, which the same lab published in 2002. To uncover this new association between EM fields and asthma, they simply checked the medical records of the kids that stayed within the same health service. This, to me, begs a question: how many different conditions did they look at? Did they also check whether the higher EM field recordings lined up with higher incidence of heart disease, or childhood cancer, or dyslexia, or eczema, or lactose intolerance? Did all of those come up negative? This information is not given, but it affects the results.
It’s like tossing a coin five times – and for this analogy I thank Prof Will Stewart of Southampton, with whom I discussed the Li paper for work. If it comes up heads five times in a row, that’s a pretty unusual result and it might make you think something other than dumb luck is involved. Something’s afoot! Check if the coin is heads both sides. See if it’s weighted. Write a paper for Archives of Coin Tossing Research. If, however, you take ten different coins and toss each one of them five times, it is much less surprising when one of them falls heads-up every time. Every time you pick up another coin, or scrape the medical records for another association with your magnetic field measurements, you ought to raise your threshold for deciding whether or not the results are down to chance. (There are established methods for this, like the Bonferroni correction, whereby if n hypotheses are tested, the p value cut-off for ‘significance’ is divided by n.) That’s why it’s important to acknowledge that the data came from a totally different study – which the authors do – but it’s even more important to know how heavily they have been trawled for significant links.
Perhaps I’m too cynical and it’s fair enough to conduct more than one study using a set of data that has taken lots of time and effort to acquire. Why should I suspect repeated trawling? At this point I draw your attention to De-Kun Li’s track record – because the band that brought you ‘EM Fields May Give Your Unborn Baby Asthma’ was behind such other hits as ‘Painkillers Might Cause Miscarriage’, ‘Hot Tubs May Cause Miscarriage Too’, ‘Magnetic Fields May Damage Sperm’ and of course their early trailblazer ‘Magnetic Fields Might Cause Miscarriage’. Every one of these contains a sentence like this (from the painkillers one): “These findings need confirmation in studies designed specifically to examine the apparent association.” And yet this team has never published twice on the same association. Many of these papers have proved controversial among Li’s fellow researchers and attracted concerned replies. Those include criticisms of experimental design, sampling and statistics; generally there is an impression that Li is willing to alarm parents with over-egged, flimsy evidence.
This willingness is apparent in the text of the latest paper. Apart from the ‘science’ itself, there is an awful lot of nonsense. In the introduction, although (as discussed) the meters only recorded mains power-derived EM fields, Li drops disingenuous, heavy hints about mobile phones and wireless networks. In the discussion section, things gets positively rabid: “Because EMF exposure is ubiquitous and exposure to it is involuntary, these new findings have important public health implications.” The paper creates an impression that a 13-year study was set up specifically to test a worrying, justifiable hypothesis: asthma is on the up, so is the use of electrical equipment. “This parallel increase in EMF exposure and asthma prevalence in the past several decades warrants exploration.” Really? What about any of the other things that have increased in the last several decades? Maybe pregnant women should stay away from contact lenses, and probiotic yoghurt, and sunscreen, and yoga.
It sounds to me like academic attention-seeking. I don’t have a problem with researchers who are junior, or who come from lesser-known universities, kicking up some dust and making a name for themselves. The world of science is unfairly weighted towards senior figures, mostly ageing men at very well-known universities, and the grants-beget-grants problem is something we’ve discussed on several Nerd Alert podcasts. I do object however – indeed, I strenuously object – when attention or promotion or funding is sought by flat-out scaremongering. That, to me, is what this paper is. Weak evidence for a scary link without any proposed biological mechanism, most probably explained by a confounding factor – but framed as a health warning that is an absolute gift to tabloid journalists, whose trade is terminally predisposed to scaremongering.
So in this case I don’t think we can blame the journalists at all. We would all agree that when scientists publish their work they have a big responsibility to consider its impact on the public. This includes thinking about how it will read in a newspaper. Most of us have the luxury, or the curse, of working on tiny details of molecules, wavelets, pathways, angles and animals so obscure that they glaze over even the loving eyes of our own families. We will probably never stumble, blinking in surprise or grinning from ear to ear, into the media spotlight. But if you publish something that happens to be both preliminary and scary, you can’t expect journalists not to write about it. They will, and that’s part of the deal. What’s more, our friend Dr Li appears to be fully aware of this and actively courts the coverage. “The message here is that exposure to electromagnetic fields is not good, and we need to pay attention to its adverse effects on health,” he says in his institute’s press release. Somebody take me to a remote cave so I can start a family!
Leaving aside self-promoting and agenda-pushing, is there an argument for publishing preliminary work? Maybe I’m being unfair and just don’t understand the way epidemiology works. Maybe ‘hypothesis-generating’ work has its place, flagging links of potential concern, so that other scientists can help follow them up and test the new hypotheses. Even though his own group has never followed them up, Dr Li’s papers all state quite plainly that more research is required.
Can you see the link? Me neither. More research needed.
The problem is that you can’t use your particular study to call urgently for more work on a potential hazard if a pile of previous studies have already failed to support that hazard. The bar is higher. It’s like finding evidence ‘consistent with’ ESP or mindreading – where countless other results, not to mention common sense, are already stacked on the other side and you can’t even imagine a mechanism for the effect you’re reporting. Sure, you might be fighting the established wisdom, which is laudable and perhaps brave, but you’re also fighting a lot of other evidence. Your findings need to be really, incredibly, insanely robust and probably independently replicated before you claim they ‘require further exploration’ or some other hand-wavy, non-committal warning.
Actually, butt-covering punchlines like ‘more research needed’ and ‘requires urgent investigation’ are brilliant, aren’t they? You can make it sound like a pressing item on the scientific agenda – a story, if you will, that no parent should miss – while simultaneously distancing yourself from making any actual health claims based on your own data.
Forgive my rant. I’m spent now. This was not intended to be a rambling, laboured, utterly disproportionate smackdown of a relatively insignificant piece of work. I wanted to deliver a little war cry against double-dipping. And against scaremongering – not by the media or PR types but by researchers themselves. When next you read about a study that suggests a ‘link’ or an ‘association’ between something (EM fields, hot-tubs, drugs, guava juice) and something else (asthma, miscarriage, autism, hairy toes) look for when and why the data were originally collected. If lots of possible associations were tested using the same data, or the authors don’t make this information clear, be wary. And hardwire your loudest alarm bells to jangle at the words ‘requires/needs/suggests more/further/urgent/extended research/investigation/confirmation’. There’s always more research needed and if these data can’t stand up alone, then they shouldn’t have been let out to play.
We all want to make the most of the experiments we do. But if you take one set of data and hurl other variables at it until something sticks, all you are doing is asking questions – not answering them.
by jonathan
HA! Especially considering that out in the real world, what happens with said wobbly conclusions is that they get reported (with no nuance) in fine rags with the word “Sunday” in the title and no capacity or real desire for sane reporting, get read (and clipped) by fine grandmothers with no capacity and little desire for sane discourse, and posted to pregnant granddaughters, who, educated and sensible or not, will get increasingly uneasy unless militant husbands screen all letters from said grandmothers. Someone should do a study on THAT – effects of maternal stress on fetus development, caused by hysterical reportage on fetus development. I’d pay money to read that.
Not being much of a consumer of scientific research papers, my primary beef with this sort of thing is – as you say – having to endure the inevitable scaremongering from “journalists” (in many cases, not my idea of what journalists should be) whose stories nearly universally feature the word “could” in their main hypothesis. If I read another story about how potatoes COULD be lethal, based on some pillock who’s done a study on something else and noticed how many people get run over on their way to do grocery shopping, I’m going to make a loud disapproving noise.
I here you Jason! Though I do think that in many cases the scientists themselves, plus journal editors and press officers, have to take some responsibility when those headlines happen. As for the ‘could cause [x]‘ thing, don’t even get me started on ‘scare quotes’. They’re almost never a direct quote – they just seem to be a little flag that means ‘even we don’t believe this bit’. Mmmmph.
I appreciate why the double-dipping is a bit irritating, but I’m okay with it. All scientists have experienced mining their data for interesting effects. It’s understandable – years of work go into a study and you have to publish to get more funding, so you might as well publish anything, whatever it is, that you can find in your data. If it’s just a fluke, then no one will repeat it and your fluke will be forgotten and dismissed.
As you suggest, this douche-bag’s douchery comes from his blatant attempts to scare-monger in public with his result. If he had simply published this and when contacted by the press said “We’re not sure what to make of this result. Parents shouldn’t worry yet as there’s other evidence that suggests it’s not a problem. But, our result indicates that further research wouldn’t be a bad idea.” then I think we’d all agree it wouldn’t be so galling. Like you imply, it’s scaremongering *by the researcher* going on here, which is really, really lame.
Do you really think it’s OK to publish a fluke when you know it’s almost definitely just a fluke? Or even if you’ve deceived yourself into thinking it’s for real, isn’t that when the reviewers should step in? It definitely happens – goes some way to explaining the ‘decline effect‘ – but I think it’s a problem and the stats should take fishing into account. Ben Goldacre's column today talks about the problem of false positives in the brain scan literature; worth a read.
In any case, I’m sick to my teeth of reading health-risk papers which are at best ‘hypothesis-generating’ and at worst just plain crap. Terrible way to be a scientist! Erodes trust, too. When someone finds a genuine health problem, we’ll be so weary of the bollocks that nobody will believe it…
The question is Jonathan: how do you know when it’s a fluke? I agree that there are some studies (maybe including the one above) where you know it’s almost definitely a fluke. But, I don’t think it’s easy to determine in most cases (e.g. cases not involving mental powers that fly in the face of our current understanding of the nature of time and space). Also, reviewers are almost definitely not the right ones to determine it. It takes knowing the experiments in depth and all the data in its rawest form to make the call, I would say. Instead, I think we should get comfortable with the occasional fluke being published, and instead focus on two more pressing issues:
1. The unpublishability of negative findings. Arguably, the decline effect (and the problem of false positives in brain scans) all come down to the issue of negative findings being hard to publish. If journal editors were more willing to accept negative papers (and if reviewers were more willing to not always chalk negative results up to experimental flaws) then I imagine the decline effect would dissipate quite a bit.
2. The bullshit reporting of health news in the media. If I could pass a law regarding science reporting in the media, it would be this: no newspaper is allowed to publish an article based on a single study. The place for people to find out about single studies is in the journals that published them – where they can get all the details that allow one to judge a single study. Newspapers should stick to reporting that which has been replicated. Yes, some false things may still slip through into the public consciousness – welcome to the world in all its messy glory, eh? The main point is that it is the constant oscillations in evidence towards the truth that are an integral part of science which cannot be captured by reporting on single studies.
[...] came across this great post by Jonathan Webb on the Nerd Alert blog. It’s a witty commentary on a recent ‘scientific’ [...]