Journals cited by Wikipedia: A compilation of over 3M citations
In 2009, I had an idea I thought was pretty neat. What if we looked at all the |journal= parameters of citation templates? Nearly 14 years later, it's more than time to share what that idea became. Here, then, is a historical tour of one of Wikipedia's interesting little secrets.
The idea was born out of a desire to understand which journals were highly cited on Wikipedia, so WikiProject Academic Journals could prioritize work and focus on the in-demand journals and redirects. I figured we had hundreds of thousands, if not millions of |journal= parameters – we might as well compile them and see what comes out of it. This was the basic idea behind Journals cited by Wikipedia, or WP:JCW for short.
After a bot request, ThaddeusB tackled the task with WikiStatsBOT (BRFA), and the idea began to take shape (albeit in a crude form). The bot would download a dump of all Wikipedia articles and process it into three searchable sub-compilations: an alphabetical one, comprehensively listing all the |journal= parameters, a most popular journal listing, and a most popular missing journal listing.
Sample outputs of the first decently-accurate run (30 June 2009):
The initial output was crude and inaccurate (especially before the above date), but it was good enough to get us started. WikiStatsBOT was quickly improved to clean up some entries, have better accuracy, and present things in a more appealing and useful way. After the early kinks were sorted out, about 355,000 citations were covered. The most popular journal entry was J. Biol. Chem. (with 17,543 citations), and the most popular 'missing' journal entry was Genome Res. (with 10,190) – Genome Research actually existed, but the redirect from Genome Res. hadn't been created yet.
ThaddeusB continued to run the bot every now and then. Its first run was in June 2009, followed by others in July and August of that year. Things were looking good: by the end of 2009, the top 300 most cited entries all had articles (or redirects), and only about 40 entries in the top ~500 remained missing. The topmost missing entry went from 10,190 citations to below 120 citations. That's a reduction of two orders of magnitude in the span of half a year! During that time, we also developed a writing guide for journal articles (which has since been greatly refined).
The next and last run didn't occur until May 2010. ThaddeusB then abruptly became inactive (a reminder that bus factors of 1 are bad), leaving us without both bot and coder. We still worked with what we had, clearing the first 500 journals by the end of December of that year.
A new bot request was made, looking for a bot to take over WikiStatsBOT's old task. I used the opportunity to bring in new ideas, and redesign some of its functionality and visual appearance. After a few days, a coder was found in JLaTondre. A BRFA was filed, and in July 2011, the JL-Bot unleashed its 0s and 1s in service of WikiProject Academic Journals.
Sample output of the first decently-accurate run (10 July 2011):
Again, the bot was quickly improved to clean up some entries, have better accuracy, and present things in a more appealing and useful way.
JL-Bot's takeover marked the beginning of what I call the "modern era" of JCW. The compilation became reliably updated, which let us focus on adding functionality and having a more polished presentation. The current update schedule is for major updates twice a month (after data dumps become available) and daily minor updates as needed. The major updates reflect the evolution of Wikipedia since the last dump (i.e. new citations, deleted citations, cleaned up citations, etc.), while the minor updates reflect smaller changes in how things have been organized since the last daily run (i.e. categorization and redirect updates, as well as bot configuration tweaks).
Over time, new sub-compilations were designed to browse the data according to different criteria.
Those can all be easily accessed through the current JCW mainpage. Browsing the archives of User talk:JL-Bot and WT:JCW will give some insights as to how each feature got implemented over time, but I must warn you that the discussions can get pretty technical. But, perhaps more importantly, the source code is available on GitHub under the MIT License, greatly mitigating the bus factor issue.
As of writing, the compilation covers about 3.3 million citations, with 1.5 million distinct DOIs, with 7,290 distinct DOI prefixes. This is nearly ten times the initial coverage we had in 2009, which reflects the expansion Wikipedia had since (both in the number of articles and in the number of citations per article). For posterity,
| Summary of the current compilation, based on the 20 July 2023 dump | |||||
|---|---|---|---|---|---|
| Most cited publishers | Citations[n 1] | Most cited journals | Citations[n 2] | Most cited missing journals | Citations[n 3] |
| Elsevier | 360,000
|
Nature | 51,000
|
The NamesforLife Abstracts | 1974
|
| Springer Science+Business Media | 286,000
|
Proceedings of the National Academy of Sciences of the United States of America | 40,000
|
Cesa News | 824
|
| Wiley | 255,000
|
Science | 37,000
|
New Zealand Journal of Geology and Geophysics | 534
|
| Nature Research | 118,000
|
Journal of Biological Chemistry | 33,000
|
The Real Estate Record: Real Estate Record and Builders' Guide | 509
|
| Informa | 112,000
|
The Astrophysical Journal | 23,000
|
Memoirs of the American Entomological Institute | 505
|
You might say, "but wait, those redlinks contains things that aren't journals!" Well, read on to find out more. I will however, take a small pause here to thank various people that helped with the development of the compilation in one way or another over the years.
First JLaTondre whose speed and skill at implementing my vision is simply unparalleled and unfathomable, as well as ThaddeusB for giving it a shot ages ago. Then, in no particular order of importance, AManWithNoPlan, the late DGG (obituary), Fgnievinski, Galobtter, IntoThinAir, John Vandenberg, Johnuniq, Mark viking, Markworthen, Randykitty, Rjwilmsi, the late Ronhjones (obituary), Smith609, Steve Quinn, TheSandDoctor, Tokenzero, and Trappist the monk. I'm sure I'm forgetting some people, and I apologize for doing so. But believe me, I have appreciated every bit of help I have even gotten.
Understanding what exactly the compilation is is important. As mentioned above, it's a searchable compilation of all |journal= parameters from citation templates on the English Wikipedia, taken from the latest database dump. The |doi= parameters are also used by the compilation to group things by DOI prefixes (the 10.xxxx/... part of DOIs). It is based on citations like:
<ref name=Bloom1969>{{cite journal |last1=Bloom |first1=E. D. |display-authors=etal |year=1969 |title=High-Energy Inelastic e–p Scattering at 6° and 10° |journal=Physical Review Letters |volume=23 |issue=16 |pages=930–934 |doi=10.1103/PhysRevLett.23.930}}</ref>It will, however, ignore named-reference repeats like <ref name=Bloom1969/>, as well as "manual" citations like
<ref>Bloom, E. D. et al. "High-Energy Inelastic e–p Scattering at 6° and 10°". Physical Review Letters, 23 (16): 930–934. doi:10.1103/PhysRevLett.23.930</ref>There is also limited support for semi-manual citations involving {{doi}} and {{doi-inline}}, like:
<ref>Bloom, E. D. et al. "High-Energy Inelastic e–p Scattering at 6° and 10°". Physical Review Letters, 23 (16): 930–934. {{doi|10.1103/PhysRevLett.23.930}}</ref>Then some cleanup and processing is done:
|journal=[[Foo|Bar]] is treated as |journal=BarMatching is not perfect, so you'll often find mismatched entries like:
| 2842 | Nature Sustainability |
|
When these are found, they can be bypassed in WP:JCW/EXCLUDE, and won't show up in the next daily run. A great deal of energy is spent dealing with bad matches, particular after new data dumps. But there will always be mismatches.
The |journal= parameter will often be misused for books, magazines, newsletters, websites, or contains wrong/extraneous data like authors/publisher/volume/page. We try to identify what type of publication we're dealing with in the most popular journals, and alphabetical sub-compilations based on categories and keywords, but no filtering is done because we want to be able to clean that stuff up when we come across it! And if we have a highly-cited non-journal, like Cesa News, well that's still good to know. If it's not notable on its own, maybe we can create an article on its publisher, Centre for Entomological Studies Ankara, and redirect Cesa News there.
Additional information on how to read the compilation can be found at the bottom of each page in the compilation, as well as on the compilation's main page.
The main historical use of the compilation was to find highly cited missing journals. That is still the case today. But so much more can now be done, particularly on cleanup:
Citation bot and JCW-CleanerBot will often be seen doing cleanup based on these compilation.
Well, the first natural extension would be WP:MCW – Magazines cited by Wikipedia. But that already exists! It was developed alongside JCW, but given much less attention; historically, {{cite magazine}} redirected to {{cite journal}}, so its adoption was much less widespread. We could also have WP:PCW (Publishers cited by Wikipedia) or WP:BCW (Books cited by Wikipedia). But those would require a great deal of curating, given those would represent entirely new datasets, with their own peculiarities. It doesn't mean they won't ever get done – just that those would represent a big design challenge. The bot code would probably be relatively straightforward to adapt once the design was clear, but I'm sure that would still have its challenges.
But for now, I hope that you'll have fun exploring the compilation, and perhaps decide you want to tackle the many invalid titles, or clean up the many proceedings cited as journals. Feel free to share your experiences with JCW or suggest improvements to the compilations in the comment section!
Discuss this story
References