A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Reviewed by E_mln_e

This report^{[1][supp 1]} from the AISciComm project discusses uses and perceptions of AI by Wikipedians in Switzerland. 159 Wikipedians, contacted through a banner and Wikimedia CH's communication channels, answered a questionnaire about their uses and perceptions of AI. This questionnaire was also answered by scientists, journalists, science communicators, science content creators (also referred to as influencers in the report) and regular citizens.

By AI, the report refers to Generative AI and Large Language Models. The tools most commonly used by participating Wikipedians are DeepL Translator and ChatGPT. The results are a fairly short and easy read, but here are the main findings, as presented by the authors and as they struck me (with my bolding).

[T]he survey suggests that Wikipedians in Switzerland are not generally unfamiliar with AI, but notably cautious about using it for their contributions to Wikipedia. While almost 90% of respondents say they use AI outside their Wikipedia work, 63.9% say they never use AI for their Wikipedia contributions. Compared with other surveyed groups, Wikipedians also report much lower AI use. (p.2)

When respondents do use AI for Wikipedia, they mainly use it as a support tool, especially for translation and language improvement. At the same time, they perceive the Wikipedia community as actively engaging with and reflecting on the use of AI [through] community discussions, [and] procedures or guidelines. (p.2)

On the other hand, most respondents did not receive any training on the use of AI (p.6).

The survey also suggests that caution is not due to a lack of competence. Many respondents say AI is easy to use, but they are less convinced that it is useful for Wikipedia. (p.2)

At the same time, respondents expect AI to matter considerably for public communication about science in general. [..] However, they do not see AI as replacing Wikipedia [...] respondents appear to take a cautious, competent, and normatively reflective position toward AI, especially where science-related information is concerned. (p.3)

Figure 1 shows that Wikipedians and scientists are far more cautious than journalists, science communicators and science influencers. This discrepancy may raise concerns about the secondary press coverage of academic research we rely on. That said, I suspect that the study could have painted a different landscape if it asked about using generative AI as a search engine, and that it doesn't account for our technical community due to its focus on content editors. It still offers an interesting window into the norms developed by Wikipedians.

The reasons underlying the development of LLM-related norms is further elucidated in a journal article, "LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities"^[2] by Moyan Zhou, Soobin Cho and Loren Terveen at University of Minnesota. They approach the subject qualitatively, through 16 interviews^{[supp 2]}. Editors primarily used ChatGPT. The paper seeks to answer three questions:

1. How does using LLMs influence the ways editors contribute content?
2. What strategies do editors leverage to conform to community norms for LLM-assisted contributions?
3. How do other editors in the community respond and engage with LLM-assisted contributions?

They highlight the paradox that LLMs may be a very helpful tool for experienced editors but do a disservice to newcomers:

Experienced editors enhance their participation through LLMs, as they expand the range of their contributions in Wikipedia, leverage strategies to align LLM-generated content with community norms, and thus receive positive responses from other members. In contrast, LLMs raise the demands for new editors to participate in the community. LLMs lower the barriers for entry, and new editors tend to rely on LLMs to fill in their knowledge gaps. However, LLMs compel them to make judgments about AI-generated content, which requires skills they have not yet developed. This challenges them to produce high quality content, leading to rejections from others.

In other words, LLMs encourage new editors to tackle complex writing projects before understanding the community and its norms. This creates a double frustration for experienced editors who have to edit or correct AI-aided content, and for newcomers who immediately receive negative feedback. This alone might explain why Wikipedians, due to the collaborative nature of their writing, are more cautious about the use of LLMs. However, experienced editors using LLM received more positive feedback about their contributions.

The paper also offers an expanded categorization of use cases for LLMs:

The results also point to a change of role for editors, from content writers to evaluators, verifiers and editors of the content generated by LLMs. It further argues that "the general sensitivity and confusion around LLMs may stem from a lack of consensus on what it means to use LLMs for editing", which aligns with the report reviewed above. It is also notable that editors participating to this study do not report using AI to generate images, while some reported that they did in the survey reviewed above.^{[supp 3]}

The article concludes with a discussion of the "paradox of participation" highlighted by the study, which should be of interest to those working on retention of new editors:

Our findings reveal a paradox of participation: LLMs simultaneously lower barriers to entry while increasing the demands of contributing, especially for newcomers who already struggle to engage with the community.

We discuss this paradox of participation in three interrelated elements:

• LLMs interrupt traditional learning pathways for newcomers that support gradual skill acquisition for newcomers.
• LLMs shift the focus from peripheral tasks to editorial judgment, requiring newcomers to make normative decisions before developing core competencies.
• As a result, LLMs exacerbate a participation divide, enabling experienced editors to thrive while marginalizing newcomers.

These remarks align with our current approaches to support newcomers. The newcomers homepage with recommendations on edits of progressive difficulties. Editathons organizers were once generally focused on creating whole pages but are now encouraged to focus on article improvements.

The ""LLMs in Wikipedia" paper concludes with design recommendations for tools aiding new editors, which the same researchers (Moyan Zhou, Minzhu Zao and Loren Terveen) have developed and reported on in "WikiCoach: Scaffolding Learning for Novices in Online Apprenticeship Communities"^[3]. The authors implement a chat interface taking into account the editing experience of the user, which can be configured for before, during, and after editing. Instead of generating text, it provides examples, feedback and reflective questions. It is built with gpt-4o-mini, and used retrieval-augmented generation to use a database of Wikipedia policies to inform answers. Their study compares it to WikiAnswer, a chat window using the same GPT model without further instructions.^{[supp 4]} It differs from previous research on harnessing LLMs for Wikipedia editing, such as Stanford's STORM, by focusing on in situ recommendations and examples, and generally on how to encourage learning.

"participants wanted LLMs to fill knowledge gaps directly and optimize for efficiency, reflecting expectations shaped by ChatGPT-like systems where success is equated with receiving answers. However, what novice editors need in order to participate sustainably in Wikipedia differs from these immediate desires: effective contribution requires learning norms, exercising judgment, and engaging in iterative sensemaking. This mismatch surfaces a critical process–product tension, in which learning opportunities embedded in the process are bypassed when LLMs assume full responsibility for generating solutions."

The study, with 28 participants with 200 or fewer Wikipedia edits, focuses on the user experience rather than the content produced. The authors analyze how participants used both tools in terms of number, timing and length of questions, number of edits and actions around sources, along with self-reported standard learning and user experience surveys. This is appropriate given the stage of research evaluating the reception of the tool and whether users' behaviors better align with editing policies. But it does not tell us how other editors evaluate the content produced, and therefore if it helps reducing or eliminating the paradox of participation the authors previously highlighted.

The results suggest a better user experience, but the most salient findings pertain to learning. While participants report similar felt learning about the subject's topic with both tools, they report much higher learning about Wikipedia policies and norms with Wikicoach, along with higher self-efficacy and ability to plan. The system also appears to encourage more detailed questions and more thorough verification of sources (see Table 1). In other words, the results suggest Wikicoach shifts participants' attitudes towards community norms, which also improves user's experience. Overall, this is a promising tool to support less experienced editors, though an evaluation of the content produced with the tool will tell us more. The authors align their work with a strand of Human-Computer Interaction research highlighting that "sometimes the more helpful AI is one that does less for the user."^{[supp 5]}

Reviewed by Tilman Bayer

This 2024 paper^[4] by Kaylea Champion and Benjamin Mako Hill (published in New Media & Society)

"... use[s] a detailed longitudinal dataset from English Wikipedia to show that more experienced contributors—including those who contribute without an account—tend to contribute to underproduced goods. A within-person analysis shows that contributors’ efforts shift toward underproduced goods over time."

The authors are motivated by the question "how well [Wikipedia] meets the needs of its users". They start out by recapping a decade-old paper that had identifed a "Misalignment Between Supply and Demand" of English Wikipedia articles (see also our review at the time: "Popularity does not breed quality (and vice versa)"):

Warncke-Wang et al. (2015) found that over 40% of views to English Wikipedia were to articles that were lower quality than one might expect given their popularity. Countries, religions, LGBT (lesbian, gay, bisexual, and transgender) topics, psychology, pop and rock music, the Internet and technology, comedy, and science fiction were found to be disproportionately affected. [...]

To quantify misalignment, Warncke-Wang et al. (2015) propose that if a system is aligned, goods in the highest demand should be the highest quality while those in the lowest demand should be the lowest quality. When quality is low relative to demand, goods are described as underproduced. When quality is high relative to demand, they are overproduced.

Adapting the approach of that 2015 paper and some of their own earlier work on open source software, the authors construct a quantitative "underproduction factor" for Wikipedia articles. An article's popularity is measured via pageviews, and its quality by using the Wikimedia Foundation's ORES score ("a machine learning-based quality measure that reflects the structural characteristics of articles, such as the presence of references, section headings, and inter-wiki links"). Articles are ranked by popularity and quality (on a monthly level), and an article's "underproduction factor" in a given month is then calculated as

the negative log of the ratio between quality rank and popularity rank. Both ranks were assigned such that high quality and high popularity receive high ranks, while low quality and low popularity receive low ones. A perfectly aligned article—one with the same popularity and quality rank—would have an underproduction factor of 0 since log(1) is 0. An overproduced article of high quality and low popularity would have a negative underproduction factor. An underproduced article of low quality and high popularity would have a positive underproduction factor.

The paper's main research questions ask how editors' experience levels relate to the "underproduction factor" of the articles that they edit. Experience is quantified using an editor's edit count (at the time of editing the article).

Analyzing this data using severel hierarchical regression models, the authors find evidence for two of their hypotheses:

[Suppported:] Hypothesis 1 (H1). Individuals with less experience will contribute to less underproduced goods than individuals with more experience. [...]
[Suppported:] Hypothesis 3A (H3A). An individual will shift toward underproduced goods as they accumulate experience

The analysis also includes contributions made without an account (known as IP editing during the timespan analyzed in the paper: 2007–2021). Champion and Hill have long been vocal defenders of Wikipedia's practice of allowing such edits, e.g. in 2019 as the authors and first signatories of a statement on Meta-wiki compiling arguments in favor of it. Their second hypothesis in this paper would have yielded an additional argument:

[Not supported:] Hypothesis 2 (H2). Contributing without an account is associated with more underproduced goods than contributing with an account.

The paper offers a theoretical motivation for this expectation, observing that "contributing without an account makes exchange of social rewards more difficult" and IP editors might thus be more motivated by public interest instead. However, this hypothesis was not borne out by the data ("Overall, [our] results contradict our proposal in H2"). Likewise, the authors only found "mixed results" for the analogue of H2A for IP editors:

[Not supported:] Hypothesis 3B (H3B). An individual not using an account will shift toward underproduced goods at lower experience levels than one using an account.

In any case, an obvious problem with relying on the number of edits previously made under an IP address as a metric for the editing experience of the person making another edit under that IP is that they might have well have previously edited using different IPs or account. The authors acknowledge this limitation ("some IP-based revisions are likely to be authored by experienced Wikipedians"), but don't offer a convincing defense of their choice in this regard.

In the paper's extensive "Discussion" section, the authors take their quantitative findings as basis for many interesting (even if largely somewhat speculative) remarks and suggestions for future research. For example, they point out a relation to the old debate of whether Wikipedians are "born" or "made":

Our results in H1 and H3A suggest that as they increase in experience, contributors tend to shift their attention to underproduced content. This is consistent with the "made" argument that accumulating experience is associated with editing underproduced articles and, presumably, with increased exposure to socialization. Our results in H2 and H3B call this interpretation into question. Socialization suggests a social process, but contributors who participate without an account have less opportunity to build a social identity in Wikipedia.

As potential answers to the question raised in the paper's title ("Countering underproduction of peer produced goods"), the paper offers that

One approach to addressing underproduction is to increase retention overall. Another approach is nudging contributors toward more severe areas of underproduction earlier: both when they are newer to the platform, and more quickly in response to emerging needs.

As examples for implementing this "nudging" approach, the authors name recommender systems and "Creating interest groups within a larger project" such as WikiProject Women in Red, WikiProject Women Scientists, and WikiProject Vital Articles. (The latter might indeed already have a measurable impact at this point, cf. our coverage of two more recent publications: "Women in blue? '[T]he trend reversed in 2022' and female biology professors are now more likely to have a Wikipedia biography than male ones".)

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

Compiled by Tilman Bayer

From the abstract of this paper (presented as a poster at the annual AAAI conference in January 2026):^[5]

"[...] we propose WikiMAG, a multi-agent guided framework for generating structured Wikipedia-like articles. This framework employs a collaborative multi-agent mechanism to orchestrate the creation process, featuring three synergistic core components: Progressive planner first constructs the coarse-grained outline framework and then annotate fine-grained types for outline units, encompassing narrative, timeline, and table formats; Reflective inspector dynamically curates high-quality references via multi-round interactive feedback, thereby enhancing the authority and relevance of citations; Versatile writer integrates fine-grained outline details and high-quality reference information to generate information-rich articles, incorporating the three annotated formats. We evaluate WikiMAG on two public datasets, FreshWiki and WikiGenBen, across outline, writing, and verifiability dimensions. Compared with the best baseline method [cf. STORM/Co-STORM], our method achieves an average improvement of 6.73 points and 4.39 points in Heading Soft Recall and the METEOR metric (a machine translation and text generation evaluation metric) respectively, and an average increase of 16.84 percentage points in Citation Rate."

From the abstract:^[6]

"We introduce a scalable framework that decompresses scientific reasoning, constructing a verifiable Long Chain-of-Thought (LCoT) knowledge base and projecting it into an emergent encyclopedia, SciencePedia. Our pipeline operationalizes an endpoint-driven, reductionist strategy: a Socratic [LLM] agent, guided by a curriculum of around 200 courses, generates approximately 3 million first-principles questions. To ensure high fidelity, multiple independent solver models generate LCoTs, which are then rigorously filtered by prompt sanitization and cross-model answer consensus, retaining only those with verifiable endpoints. This verified corpus powers the Brainstorm Search Engine, which performs inverse knowledge search—retrieving diverse, first-principles derivations that culminate in a target concept. This engine, in turn, feeds the Plato synthesizer, which narrates these verified chains into coherent articles. The initial SciencePedia [in Chinese, available at https://www.bohrium.com/sciencepedia ] comprises approximately 200,000 fine-grained entries spanning mathematics, physics, chemistry, biology, engineering, and computation."

From section 5.2 ("Advantages over LLMs and Traditional Encyclopedias"):

"This methodology provides significant advantages over both querying a general-purpose LLM directly and relying on human-curated resources like Wikipedia. Compared to a direct LLM query, our approach ensures greater depth, as demonstrated by the higher knowledge-point density in our generated articles. [...]

SciencePedia’s automated approach also addresses fundamental limitations of human-curated wikis. It overcomes challenges of scalability and language parity [1, 2], as the pipeline can generate high-quality pages consistently across languages, unlike the volunteer-dependent model which often results in coverage disparities [3]. It also provides greater explanatory depth, moving beyond the “checklist” style of some articles to unfold the underlying reasoning process. Lastly, where a human-written page is limited by the author’s breadth of knowledge, our search-driven method ensures a systematic and comprehensive coverage of cross-domain applications, providing a more complete view of a concept’s role in science [47]."

From section 5.3 ("Writing Philosophy and Current Scope"):

"To maximize accessibility, the writing style of SciencePedia is inspired by Richard Feynman’s “Feynman Lectures on Physics” [...], focusing on advanced popular science that is intuitive and insightful while minimizing dense formulas and jargon. This pedagogical approach is intentionally distinct from the typically drier style of Wikipedia, though our framework is flexible enough to produce more technical articles by adjusting the authoring prompts."

See also thread by one of the authors

From the abstract:^[7]

"We [are] studying the deployment of neural machine translation across more than 100 Wikipedia language communities. Leveraging rich, fine-grained data and exogenous variation from a natural experiment, we uncover the “AI democratization paradox,” where the technology simultaneously drives democratizing and concentrating forces. AI lowered barriers, leading to a substantial increase in content creation across diverse target languages without sacrificing quality or readership. However, the benefits were concentrated: well-resourced communities captured disproportionate gains—three to four times larger than mid-tier editions. Whereas editors actively leveraged AI to address representation gaps, translating female biographies at twice the expected rate, structural constraints still limited the impact in high-need areas."

See also an accompanying "thread on Wikipedia + AI translation" by the first author.

The paper acknowledges support from the Wikimedia Foundation, in form of a 45,840 USD grant from the 2022-23 round of the Wikimedia Research Fund (see also the research project page on Meta-wiki: "Can Machine Translation Improve Knowledge Equity? A Large-scale Study of Wikipedias across more than 300 language editions"). The authors had already published a related paper in 2024, see our coverage: "The 2019 integration of Google Translate made Wikipedia editors more productive".

From the abstract:^[8]

"Large language model (LLM)-based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet their measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule. These details are frequently under-specified, making it difficult to compare results or reproduce reported baselines. We present SimpleWikiSearch, whose corpus construction, retrieval stack, tool contract, and evaluation protocol are explicit and runnable. The environment starts from a full English Wikipedia dump, cleans and chunks the corpus, builds keyword and dense retrieval indexes, and exposes a minimal tool interface [...]"

From the abstract:^[9]

"We conduct a large-scale political bias study on Grokipedia and Wikipedia, analysing 1,394 article pairs describing members of government for neutrality along nine expert-coded ideology dimensions employing four LLM judges, Grok, Claude, Mistral, and DeepSeek. As the LLMs could themselves be biased, we also investigate patterns in their judgments. We find all LLM-judges, including Grok, to rate Grokipedia less neutral than Wikipedia. Both encyclopedias are rated as portraying politicians favourably overall, but towards different ideological groups. Grokipedia particularly favours economically right-wing politicians and penalises socially liberal ones, while Wikipedia is rated as favourably biased towards the latter

(See also our coverage of earlier research comparing Grokipedia and Wikipedia, in particular: "Comparing Political Framing in Wikipedia and Grokipedia" and "Comparing comparisons of Grokipedia vs. Wikipedia by three different research teams")

Supplementary references and notes: