Two years ago I posted about a story that had gone viral on Facebook, in which the World Cancer Research Fund claimed bacon would cause stomach cancer. When I did actual due diligence to see where they came up with that claim, I found no evidence it was backed up with anything remotely akin to science. In fact the little bit I could discover on Google suggested that the WCRF either mostly or exclusively does meta-studies, analyzing trends in other people’s research, which is as scientifically valid as drawing theories out of a hat and proclaiming one of them true. Meta-studies are not science: they do no new research of their own, and combine the results of multiple studies which may have radically disparate methodologies, sample validity, etc.
There’s a new article today in the Mirror, a UK paper, claiming no amount of alcohol, bacon, or sausage is safe and all of them increase your cancer risk. Inside the article they also mentioned processed meats, the great hobgoblin of the WCRF’s last scare, and sure enough the WCRF was at the front and center of this piece. Once again there’s no citation of science done. This time they didn’t even bother to hint at what kinds of approaches they even took to reach this conclusion. For that I blame the Mirror, for publishing an article supposedly on a science topic but not asking for even four words to describe the methodology behind this bold, dare I say outrageously asinine (I dare), conclusion.
Any time an article tells you that supposed experts in a field came to a shocking conclusion, and doesn’t tell you how, you know two things right off the bat: The journalist utterly failed at their job of reporting the salient facts, and the “experts” probably are not experts but are more likely to be an advocacy group putting out a press release to drive a narrative.
Does the shoddy quality of this article prove the WCRF is nothing more than an advocacy group pushing junk science? No. But if they’re not, they should be furious with the author of the article for making them look that way, because holy crap do they look that way.
Real scientists want people to understand how they reached their conclusions. Real scientists lay their data and methodology bare for the world to see, and beg others to poke holes in their findings so they can refine their work and get closer to the truth. Any time a supposed scientist goes out of their way to avoid replication, bias peer reviews, or push a conclusion instead of continuing to question it, they’re not doing science. And any time a science journalist fails to spot such problems, they’re not doing journalism.
Let’s name names! The reporter behind this article is the Mirror’s health and science correspondent, Martin Bagot. Here are my questions for Mr. Bagot:
- What was your source for this article? Was it, as I suspect, a press release? And if so, didn’t it occur to you to question anything it said?
- What specific research did the WCRF site in the source material? Why was absolutely no mention of that specific research, or lack thereof, made in the article?
- Why does the word “study” appear nowhere in the story until a quote from the head of health of Bowel Cancer UK, and why is said study never mentioned in the article outside of that quote? (See also question 2.)
- Speaking of Bowel Cancer UK, why is it not disclosed or even mentioned that they are a charity group with a particular agenda? A group looking to get more funding, albeit obviously for a worthy cause, has a vested interest in calling special attention to alarmist findings, does it not? Which could also be said of the WCRF itself, could it not?
- As a science correspondent, isn’t it your frelling job to include basic information regarding the research in question, including its methodology, sample sizes, and so on? If there was an actual research study done, why was this information left out of the article, and did you even try to find out? Or, if this article pertains to a meta-study, the scientific equivalent of getting your horoscope from a puppet, why was that never mentioned?
- Why didn’t the “No amount of ____ is safe” claim not immediately peg you that this claim warranted extra skepticism? Or was that extremist wording yours, or the Mirror’s, rather than the WCRF’s?
- How in the world did your editor let this story through without addressing, or rendering moot through due diligence, every single one of the previous questions?
So let’s assume the WCRF’s findings here are on the up-and-up. Let’s assume for the sake of argument that it’s the result of rigorous valid science, leaving aside that I said in my previous post that there is almost no such thing as valid science in the field of linking specific dietary items to cancer, for reasons that are obvious to scientists and those who hold the scientific method in high regard. Let’s pretend this study is beyond reproach. In that case, Martin utterly botched the story by failing to describe, in overview, how the WCRF reached its findings. I mean that’s the bare minimum he’s supposed to do in every article about a scientific finding: give us a brief glimpse of the science behind it, so we can see if it’s thoroughly backed up or if it’s suspect.
Now let’s assume the far more likely scenario, that the WCRF is relying on stupid meta-studies and putting out an alarmist press release. Martin still dropped the ball pretty heavily, by repeating the source material uncritically. As a science reporter you do not get to simply say “Experts said X” and not tell us their justification—and if you don’t know their justification, you push for details or you don’t run the story! Turning a skeptical eye to all research claims is not only his job, but the job of all scientists everywhere.
Interestingly, the one relevant thing Martin mentions in the article is that the WCRF included stomach cancer among other cancers on their list for the first time with this new report, even though two years ago I was seeing them say the exact same thing. This news is not new, even if it’s true. Maybe the change to their guidelines is a new thing, but the supposed link to stomach cancer is not. Also, this fact is trivial compared to the more serious concerns about where the conclusions came from and how; if the article was kept short to fit a certain word count, then this was the wrong information to include.
And really, this part gets my goat the worst: “No amount of ____ is safe.” That’s such a ridiculously extreme statement it doesn’t pass the sniff test. Maybe the only thing that statement might be true for is aerosolized plutonium, the deadliest substance known to man. For everything else, even tobacco, it’s flat-out wrong. Whenever you read a statement like that, if your BS detector doesn’t start screaming then you need to get a new one because yours is broken beyond repair. That’s not a statement from science, it’s a statement from someone trying to use shock to push an agenda. Whether that agenda is to push everyone into eating vegetarian or just drum up more funding for a cause or whatever, I have no idea; but whatever it is, that statement was not a conclusion someone came to through research. (For clarity’s sake, I don’t know whether that statement came from the WCRF or it’s a muddled echo dumbed down by Martin or whoever wrote the headline. But if it came from the WCRF, that’s all the more reason Martin should have been on the ball.)
Oh, I know, I know: maybe I’m being too hard on Martin. Only I’m not. If you’re a science correspondent for a major publication you owe it to your readers and your employers to get the whole story, as much as possible, every time. If your publication for some reason imposed untenable bounds on your ability to do that, shame on them, but shame on you too for letting them. And none of how he’s screwed up is malicious, but it sure is lazy. Digging for the deeper story is important, even if you’re doing something as picayune as a human interest story on a dog who sings and plays piano. This is about cancer.
And as for the WCRF, suffice it to say what I’ve seen of them across two articles and a bit of lightweight Internet sleuthing has suggested to me they’re not doing science; they’re doing meta-studies. This is only a hunch, not an accusation, but I suspect they have a bit of confirmation bias when it comes to that, where the more extraordinary a result appears to be the more they want to believe it, so they can put out press releases that bonehead science reporters will repeat, that will remind the public of their name and get more funding kicked their way. I don’t blame them for wanting to push funding or hyping extraordinary results, because cancer is serious and research is important, and I’m sure everyone in that organization has their hearts in the right place. But it’s precisely because good research is so important that if they are doing meta-studies, that crap needs to end. Either do real science or direct your research funds towards people who are.
Science is important, and so is reporting on it correctly. The whole scientific endeavor has had quite enough trouble lately as it’s come to light repeatedly that many researchers—doing or at least purporting to do actual research, not meta-studies—have had to retract their papers for serious flaws ranging from severe biases to outright scientific fraud. So why hasn’t science reporting, in major publications at least, not gotten the least bit better as we recognize just how fraught with inaccuracy, mistakes, and misinformation the discipline has become? You don’t have to have the skeptical bona fides of James Randi to realize there’s a problem with whatever was used to source the article, and try to get to the bottom of it in an objective way.