RetractionBot is back to life!

Special report

RetractionBot is back to life!

Contribute —

By Headbomb

A bit of history, context, and what you can expect to see in articles

RetractionBot, from pre-history to v2

reliability

RetractionBot is back to life!
8 June 2024

Citation tools for dummies!
1 August 2023

Re-righting Wikipedia
29 November 2020

The Wikipedia SourceWatch
31 March 2019

Philosophers analyze Wikipedia as a knowledge source
23 February 2009

More articles

Dispatches: Reliable sources in content review processes
26 June 2008

Poll finds people think Wikipedia "somewhat reliable"
2 April 2007

Back in 2012, Doc James made a query over at WikiProject Medicine about what sort of work could be done by a bot to find retracted papers. At the time, there was no centralized way of finding retracted papers, so Rich Farmbrough queried the PubMed database looking for retraction-related keywords (like 'retraction of publication' in the metadata). Of the roughly 4000 retractions, he found 138 that matched papers cited on Wikipedia. The template {{retracted}} was created to flag those papers, and was manually and semi-automatically added to articles.

Then in 2018, JenOttawa noticed the then newly-launched RetractionDatabase.org, a database of retractions maintained by Retraction Watch. This led Samwalton9 to code the first iteration of RetractionBot. The bot was then doing automatically what people did manually, saving everyone a lot of hassle. However, the bot only ran for a few months, and hit a snag: several Cochrane Reviews were flagged as retracted for technical reasons, while they were never retracted in actuality. The bot was put on hiatus, and Samwalton9 never got to fixing the issue.

Five years later, motivated by the slew of retractions hitting major publishers from Elsevier, Hindawi, SAGE, and many others, as well as the opening up of RetractionDatabase.org (now with nearly 40,000 retractions), I thought it would do us some good to kick the hornet's nest and see if I could interest someone in revisiting this project.

Turns out I could. Not even a week after probbing the volunteers at WP:BOTREQ, mdann52 graciously took over maintenance of RetractionBot, and the bot is now back alive, with many improvements. In particular, it now covers expressions of concerns, not only retractions, which are early warning signs that a paper might be dubious and could be retracted/in need of a major revision. This led to the creation of {{expression of concern}}, which works very similarly to {{retracted}} (see below).

What the bot does

citations

RetractionBot is back to life!
8 June 2024

Tens of thousands of freely available sources flagged
4 December 2023

Citation tools for dummies!
1 August 2023

Journals cited by Wikipedia
1 August 2023

The unexpected rabbit hole of typo fixing in citations...
31 August 2022

More articles

The bot first downloads a .csv file containing all the information in the RetractionDatabase (a 50MB download available here). Then it crosschecks retracted DOIs and PMIDs in the database against those found on Wikipedia.

If a match is found, the bot will, for example, change

...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.^[1]

...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.^[1]

1. ^{^} Restrepo-Arango, Marcos; Gutiérrez-Builes, Lina Andrea; Ríos-Osorio, Leonardo Alberto (April 2018). "Seguridad alimentaria en poblaciones indígenas y campesinas: una revisión sistemática". Ciência & Saúde Coletiva. 23 (4): 1169–1181. doi:10.1590/1413-81232018234.13882016. PMID 29694594. (Retracted, see doi:10.1590/1413-81232018241.32242011, PMID 30698268, Retraction Watch. If this is an intentional citation to a retracted paper, please replace {{retracted|...}} with {{retracted|...|intentional=yes}}.)

It is now up to humans like you to review if this is problematic for the article. If the citation is no longer reliable, then the article needs to be updated, which could be as minor as the removal/replacement of the citation with a reliable one, to rewriting an entire section that was based on flawed premises. If the citation to a retracted paper was intentional, like in the context of a controversy noting that a paper was later retracted, you can replace {{retracted|...}} with {{retracted|...|intentional=yes}}, suppressing the red notice

...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.^[1]

What you can do

If you are interested in doing systematic work involving Wikipedia articles citing retractions, the category Category:Articles intentionally citing retracted publications will be automatically populated by {{retracted}}. The retractions that haven't yet been reviewed by a human can be found in the sub-category Category:Articles citing retracted publications instead.

Otherwise? Well... carry on as usual. But if you see one of those big red notices, don't panic. Treat it like any other unreliable source, and update the article accordingly. If a retraction paper (or one with an expression of concern) is intentionally cited, then simply follow the instructions and replace {{retracted|...}} with {{retracted|...|intentional=yes}} (or {{expression of concern|...}} with {{expression of concern|...|intentional=yes}}) to suppress the red notice.

And while these are not currently handled by RetractionBot, {{erratum}} and Category:Articles intentionally citing publications with errata / Category:Articles citing publications with errata work very similarly to the above categories for papers with errata and might also be of interest to you.

Happy editing!

← Previous "Special report"

Next "Special report" →

In this issue

8 June 2024 (all comments)

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Good idea for a bot - never heard about it or seen it in action, but I'm glad it's back up and running - kudos to those responsible! —Ganesha811 (talk) 12:41, 8 June 2024 (UTC)[reply]
See Special:Contributions/RetractionBot if you want to see some of its edits. Though the categories mentioned in the article are the best way to check what currently needs attention. Headbomb {t · c · p · b} 12:44, 8 June 2024 (UTC)[reply]
Aren't retracted articles below unreliable sources in terms of quality? With an unreliable source, we don't know if the information is good or not; with a retracted article, we know that somebody has found a major problem. No? —Compassionate727 ^(T·C) 00:48, 9 June 2024 (UTC)[reply]
That's why we're flagging them. This way we don't cite them thinking they are reliable. Headbomb {t · c · p · b} 01:22, 9 June 2024 (UTC)[reply]

Generally, but not always. Upon reading this article, I took a look at Category:Articles citing retracted publications, and clicked through to Autism therapies, a topic I don't know very much about. The relevant text in the article reads:

In the early 1990s, it was hypothesized that autism could be caused or aggravated by opioid peptides like casomorphine that are metabolic products of gluten and casein. Based on that hypothesis, diets that eliminate foods containing either gluten or casein, or both, are widely promoted, and many testimonials can be found describing benefits in autism-related symptoms, notably social engagement and verbal skills. Studies supporting those claims had significant flaws, so those data were inadequate to guide treatment recommendations.

The claim in the first sentence—that there was a hypothesis of a link between autism and metabolic products of gluten or casein—is supported by a 1991 paper. That paper isn't online, but the abstract says it is based on a study of 30 children. It doesn't really matter whether it is accurate, because it does establish what it is being cited for—that a hypothesis of the sort existed, not that it is true.

The second sentence is uncited—but it does not seem particularly unbelievable that people jumped on a hypothetical link between gluten/casein metabolisation and autism and promoted diets around it of dubious credibility. Wikipedia has an entire category on autism-related pseudoscience, after all.

The final sentence—saying that there isn't really any truth to the hypothesis—cites a 2006 review article and a now retracted Cochrane review originally published in 2008. Why was the Cochrane review retracted?

This review was withdrawn from the Cochrane Library in Issue 4, 2019, as it has not been updated since its last revision in 2008. The editorial group responsible for this previously published document have withdrawn it from publication.

They're not saying "this is debunked nonsense", they're saying "it's old and unless the article gets updated, our policy is to retract it". If an article is retracted because it's trash, then yes, that's a good reason to not rely on it. If an article is retracted because a diligent body like the Cochrane Collaboration are concerned that it might be in need of an update... that doesn't immediately make it useless. Editors have brains, and it is fine to use them. Carefully considering how Wikipedia uses it as a source and the reasons behind the article's retraction would be strongly advised in deciding whether to continue using it. If the article was retracted because it is a load of made up junk, that's a real problem. But that's not always going to be the case. —Tom Morris (talk) 16:33, 9 June 2024 (UTC)[reply]
The Cochrane review being flagged here is what I was referring to by "several Cochrane Reviews were flagged as retracted for technical reasons". Ideally, RetractionBot would leave those alone, and let User:Pi bot deal with Cochrane Reviews (see BRFA and Wikipedia:WikiProject Medicine/Cochrane update). Headbomb {t · c · p · b} 16:50, 9 June 2024 (UTC)[reply]
It's incredibly bad luck that I happened to randomly choose this as an example then. I'm glad people who actually know something about these topics are thinking hard about it. Between the rapid rise in conspiracy theories, the replication crisis and the preprint-to-tweet pipeline that flourished during the peak years of COVID, knowing stuff seems to have had a tough time recently—any effort to try and fix that that is commendable. —Tom Morris (talk) 14:17, 10 June 2024 (UTC)[reply]
I'd argue it was good luck. I didn't have room in the piece to got into the nitty gritty, and didn't really know how to summarize the issue concisely and intelligibly. But here, with more space, you summarized it better than I ever could. Headbomb {t · c · p · b} 23:10, 10 June 2024 (UTC)[reply]

(edit conflict)Tom Morris well, you've just cited the exact article which is partially why the bot stopped running in the first place and has been causing me struggles since - see User talk:RetractionBot#How to avoid an edit warring bot and (more recently) Wikipedia talk:WikiProject Medicine#Dealing with Cochrane Review retractions. The consensus over there, seemed/seems to be that the 2008 paper had not been withdrawn, but I'm still trying to get my head around that and other papers and exactly how they've handled it before I actually process Cochrane Review stuff! Mdann52 (talk) 16:52, 9 June 2024 (UTC)[reply]
I'm personally just tempted to remove all retracted notices from Cochrane Reviews and let Pi bot deal with it. But we should have that discussion at WT:MED. Headbomb {t · c · p · b} 17:04, 9 June 2024 (UTC)[reply]
Currently resisting the temptation to wikilink humans like you to Viewers like you. → FeRDNYC (talk) 04:59, 9 June 2024 (UTC)[reply]
Instead, I diddled the formatting of the example references; see edit summary for details/justification. FeRDNYC (talk) 05:11, 9 June 2024 (UTC)[reply]
Wow this is a real public service, thank you to all involved. Innisfree987 (talk) 07:56, 9 June 2024 (UTC)[reply]
Great article! Frostly (talk) 17:37, 20 June 2024 (UTC)[reply]

Keep up with The Signpost on Twitter, Facebook or Mastodon.

Home

About