 Article display preview: | This is a draft of a potential Signpost article, and should not be interpreted as a finished piece. Its content is subject to review by the editorial team and ultimately by JPxG, the editor in chief. Please do not link to this draft as it is unfinished and the URL will change upon publication. If you would like to contribute and are familiar with the requirements of a Signpost article, feel free to be bold in making improvements!
| ||||
A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.
Ashkinaze, Joshua; Guan, Ruijia; Kurek, Laura; Adar, Eytan; Budak, Ceren; Gilbert, Eric (2024-09-04), Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms, arXiv, doi:10.48550/arXiv.2407.04183
From the abstract: "Large language models (LLMs) are trained on broad corpora and then used in communities with specialized norms. Is providing LLMs with community rules enough for models to follow these norms? We evaluate LLMs' capacity to detect (Task 1) and correct (Task 2) biased Wikipedia edits according to Wikipedia's Neutral Point of View (NPOV) policy. LLMs struggled with bias detection, achieving only 64% accuracy on a balanced dataset. Models exhibited contrasting biases (some under- and others over-predicted bias), suggesting distinct priors about neutrality. LLMs performed better at generation, removing 79% of words removed by Wikipedia editors. However, LLMs made additional changes beyond Wikipedia editors' simpler neutralizations, resulting in high-recall – low-precision editing. Interestingly, crowdworkers rated AI rewrites as more neutral (70%) and fluent (61%) than Wikipedia-editor rewrites. Qualitative analysis found LLMs sometimes applied NPOV more comprehensively than Wikipedia editors but often made extraneous non-NPOV-related changes (such as grammar). LLMs may apply rules in ways that resonate with the public but diverge from community experts. ... Even when rules are easy to articulate, having LLMs apply them like community members may still be difficult."LLMs such as ChatGPT struggle with identifying neutrality violations on Wikipedia ("better than random individuals...but worse than expert editors")
With a paper title that harkens back to James C. Scott's, Seeing Like a State one might expect this paper to have something to do with political theory, and it does to an extent – an intriguing hypothesis that norms used in a community can be intentionally applied as constraints for the production of a large language model to good effect. Or, as the authors state:
Large language models (LLMs) are trained on large, broad corpora but then used within smaller communities that have their own norms. To steer models towards specific norms and values, there is a growing trend of stating high-level rules as prompts.
— Introduction to the paper
The experiment, as outlined in the abstract above, centered on "fixing" biased edits, and introduced another bit of word play: "high recall but low precision editing" (emphasis added by reviewer) is (perhaps) a new term, mixing terminology that usually applies to classifier measurements with the act of text generation which large-language models excel at. It emphasized to this reviewer the authors' intent to bring new perspectives to machine learning. In any event, the interpretation of the LLM results was that they sometimes made over-broad changes (perhaps beyond what a skilled human community member would have done with the same constraints).
The dryly put statement in the paper "LLMs may reduce editor agency and increase moderation workload" would seem to not exactly align with concerns the English Wikipedia community expressed in enacting its recent ban on incorporating unreviewed LLM outputs. This cleanup and moderation workload (including not insignificant detection of undeclared use of LLMs) were key frustrations that were expressed during the highly engaged community discussion. (See related Signpost coverage.)
Another item of potential interest to the Wikipedia community is in a door left open in this finding presented by the authors:
If future decisions to adopt, or not adopt, various forms of AI technology are made by the current community, and not readers or some other authority, could further research address what that AI would look like, in order to meet the community's requirements? There may be very human concerns about community-building at play here, such as making conservative edits to others' writing in order to preserve collegiality and the needs of the community, even when at odds with the "best" presentation of a neutral point of view. How would an AI respond to prompts to make not just correct, but civil edits?Crowdworkers prefer AI edits over human edits on both fluency and neutrality. We note that participants were not Wikipedia editors. Their judgment may be more representative of Wikipedia readers.
Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.
From the abstract:
...
From the abstract:
...
From the abstract:
...
Discuss this story
(This allows for greater visibility of discussions, makes archiving easier, and prevents discussions becoming disconnected from articles during the publication process)