A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Reviewed by E_mln_e and Tilman Bayer

This report^[1] from the AISciComm project authored Mike S. Schäfer, Damiano Lombardi, Gerta Lokaj, Xiran Liu, Daniela Mahl and Sophia C. Volk, discusses uses and perceptions of AI by Wikipedians in Switzerland. 159 Wikipedians, contacted through a banner and Wikimedia CH's communication channels, answered a questionnaire about their uses and perceptions of AI. This questionnaire was also answered by scientists, journalists, science communicators, science content creators (also referred to as influencers in the report) and regular citizens.

By AI, the report report refers to Generative AI and Large Language Models. The tools most commonly used by participating Wikipedians are DeepL Translator and ChatGPT. The results are a fairly short and easy read, but here are the main findings, as presented by the authors and as they struck me.

[T]he survey suggests that Wikipedians in Switzerland are not generally unfamiliar with AI, but notably cautious about using it for their contributions to Wikipedia. While almost 90% of respondents say they use AI outside their Wikipedia work, 63.9% say they never use AI for their Wikipedia contributions. Compared with other surveyed groups, Wikipedians also report much lower AI use. (p.2)

When respondents do use AI for Wikipedia, they mainly use it as a support tool, especially for translation and language improvement. At the same time, they perceive the Wikipedia community as actively engaging with and reflecting on the use of AI [through] community discussions, [and] procedures or guidelines. (p.2)

On the other hand, most respondents did not receive any training on the use of AI (p.6).

The survey also suggests that caution is not due to a lack of competence. Many respondents say AI is easy to use, but they are less convinced that it is useful for Wikipedia. (p.2)

At the same time, respondents expect AI to matter considerably for public communication about science in general. [..] However, they do not see AI as replacing Wikipedia [...] respondents appear to take a cautious, competent, and normatively reflective position toward AI, especially where science-related information is concerned. (p.3)

I do not believe that the results will be a surprise to most Wikimedians who have been curious about generative AI. It is however disheartening to see that the journalists and science communicators we may be trusting to provide high quality, verified information, do not seem to share our caution. That said, I suspect that the study could have painted a different landscape if it asked about using generative AI as a search engine, and that it doesn't account for our technical community due to its focus on content editors. It still offers an interesting window into the norms developed by Wikipedians.

These are further elucidated in a journal article, LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities by Moyan Zhou, Soobin Cho and Loren Terveen at University of Minnesota. This paper aims to answers three questions:

1. How does using LLMs influence the ways editors contribute content?
2. What strategies do editors leverage to conform to community norms for LLM-assisted contributions?
3. How do other editors in the community respond and engage with LLM-assisted contributions?

They highlight the paradox that LLMs may be a very helpful tool for experienced editors but do a disservice to newcomers:

Through semi-structured interviews with 16 participants who have used LLMs to edit Wikipedia, we unveil an expertise-based participation divide between editors. Experienced editors enhance their participation through LLMs, as they expand the range of their contributions in Wikipedia, leverage strategies to align LLM-generated content with community norms, and thus receive positive responses from other members. In contrast, LLMs raise the demands for new editors to participate in the community. LLMs lower the barriers for entry, and new editors tend to rely on LLMs to fill in their knowledge gaps. However, LLMs compel them to make judgments about AI-generated content, which requires skills they have not yet developed. This challenges them to produce high quality content, leading to rejections from others.