NPR and AfC – The Marshall Plan: an engagement and a marriage?: Further developments on New Page Review and Articles for Creation work sharing.
Within minutes of ACREQ being switched on, the new rule that limits creation of articles in mainspace to confirmed users, the number of totally inappropriate new pages dropped back to the level of the six-month trial.
As reported in our April issue in Special report, the switch, originally scheduled for 3 May, was urgently thrown a week earlier on 26 April bringing this six-year-long-awaited new policy finally into play.
Promised by the Foundation, based on comments made by AfC reviewers during the RfC, special envoy Marshall Miller has been looking into the way AfC works, how it contrasts with New Page Patrol, and making some suggestions which have since moved to development stage.
Miller's analysis appears to have identified issues surrounding the productivity of AfC which can be addressed with software enhancements, while some editors involved in the discussion for improvement of the system suggest that the problems of AfC are social ones rather than technical: poor reviewing and too few reviewers.
| “ | This is less about "overhauling AFC" as it is "reminding reviewers that imperfect drafts are okay" | ” |
Indeed, some are OK, but perhaps many of them are not, and a large number of them could be consigned immediately to the trash can. Many pages are dumped into the draft system by people who appear not to have the slightest intention of contributing anything coherent to a collaborative project, while others simply submit a substandard draft never to return. Since ACREQ, New Page Reviewers are increasingly moving borderline articles to the draft mainspace, thus adding to the workload for AfC but not on the scale that was feared. A dilemma faced by AfC reviewers, however, is that getting rid of even the most clearly unwanted draft is not so easy. Talks are ongoing on the possibilities of either introducing a special Criteria for speedy deletion criterion, a sticky proposed deletion à la Proposed deletion of biographies of living people, or the creation of a "Drafts for Deletion" on the lines of Articles for Deletion.
Although AfC is not the Article Rescue Squadron, many draft creators (and confirmed editors too), especially single-purpose accounts, submit their creation in Wikipedia expecting other editors to complete it or clean it up. They need to be informed up front that not only are notability and sources required, but that the article must also be appropriate for an encyclopedia, and that clean-up attempts by reviewers might not be the best deployment of their enthusiasm to rescue certain kinds of articles.
Dialogue with the creator is an intrinsic part of the AfC template system, and used well, more effective than Page Curation's message feature. Unlike New Page Review, whose principal task as a triage is to either tag articles for deletion or pass them for inclusion with perhaps some minor details needing to be addressed, at AfC the skill is in being able to sensibly recognise whether or not a new article has true validity and potential for the encyclopedia and offer some basic advice – the rest is about not being scared to keep or delete:
| “ | Clearly some amount of bravery and humility is required of reviewers as we, in some sense, take responsibility or vouch for the work of others. For me that makes reviewing more worth doing. The community is forgiving of mistakes and lapses or judgement. Failure to admit to or learn from mistakes (often rooted in a lack of humility) is what leads you down the road to sanctions.
—Kvng |
” |
| “ | I made some updates to the project page to include seven additional potential improvements that were raised in the discussion here. I also added a couple points to other sections brought up by this group.
There has been a lot of discussion (and changes already!) around the language in templates, around putting templates and comments on talk pages instead of draft pages, and on the comment workflow in general. Those things all seem to me to be related, and perhaps we might want to think about them holistically. One new idea that was brought up was "inline comments" – the idea of having a feature that allows reviewers to make specific comments to different parts of a draft. The team here at WMF thought that was an interesting idea, and I'm interested in everyone's thoughts on that idea. Being able to detect and surface copyvio sounds like it would be a particularly valuable improvement – more valuable than having ORES scores. The idea of routing or tagging drafts so that topic experts (perhaps from WikiProjects) can review them got some good discussion. Many people are positive on the idea of unifying the interfaces of NPP and AfC, but this is likely out of scope for this project, given its size and importance. |
” |
 | Graphs are unavailable due to technical issues. There is more info on Phabricator and on MediaWiki.org. |
After the New Pages Reviewer user right was created in November 2016, the effect was to reduce a mammoth backlog of some 22,000 articles down to 3,500 in just over a year, hitting what is probably the lowest level ever of exactly 700 just before the copy deadline of this month's Signpost. Prior to the introduction of the NPR user group, poor patrols and incorrect tagging were frequent; odd uses of tags and deletion criteria still occur, but at a much reduced level.
In an April 2018 article, Foundation chief executive Katherine Maher discussed similar problems, though it was noted that "longtime Wikipedia editors are able to distinguish this kind of activity, and conduct their own investigations to weed such actors out."[2] The problem facing Wikipedia in reality, however, and which the Foundation's research cannot identify, is that many of the older and more experienced editors have long since moved on from the mundane and depressing task of repetitively tagging trash. It's then largely left up to enthusiastic but inexperienced new users who still do not need special rights to apply the tags to the articles and drafts.
Automation may be able to help. Probably hinting at ORES, Maher goes on to say:
| “ | We are very interested in how AI can help us do things like evaluate the quality of articles, how deep and effective the citations are for a particular article, the relative neutrality of an article, the relative quality of an article.[2] | ” |
Notability should be as much a part of the process as it is at New pages patrol. However, this tends to be more of an issue of subjective interpretation of notability by the reviewers. Unless they know them by heart through years of creating articles or patrolling them, no one knows the mass of notability guidelines properly or has even read them until push comes to shove. Just not having sources in the article is not a reason for a lack of notability if credible claims of notability are expressed in the article. That said, a raft of sources needs to be carefully examined; chances are that the more references that come with a new article, the majority of them are just Internet barrel-scraping, and what's left is barely reliable. It shouldn't necessarily be the reviewer's job to go searching for sources. Creators need to be pointed to the instructions for reliable and verifiable sources and asked to go back and do it themselves. In this respect the Article Wizard could be improved – currently it directs the creator to Citing sources which is a mind bending page with a steep learning curve for a newbie. A simpler version needs to be written.
Among the other suggestions were the possibilities of:
The third option has been chosen by the Foundation as the one they can address within the resources available at this time. Notwithstanding:
Replying to Miller's question – "Would you say that the biggest benefit with this idea would be 'improve communication between reviewers and authors to decrease iterations', 'increase the speed/ease that reviewers can do their workflow', or both?" – Legacypac says that it "[d]efinitely improve[s] communications between reviewers and authors and between different reviewers. Better communication improves workflow in all contexts."
Based on Marshall Miller's findings, a summary of AfC's challenges, goals, and ideas for improvement, Community Tech and volunteers are already collaborating on developments bearing in mind that knowing what is needed and writing the code for it are very different specialisms. Miller has been posting regular progress updates and asking for community feedback. The collaboration has emphasized creating separate list of drafts in the New Pages Feed for better organisation of the way AfC agents select the drafts they wish to review and finally accept or reject. The entries in the feed will be flagged by ORES for copyright violations.
Miller's list of 18 improvements included: "take steps toward a shared interface between AfC and NPP, since those processes are similar in many ways" and "automated checks for copyright violation as part of the helper script" and "rotate reviewers on repeat submissions instead of routing repeat submissions to the same reviewer".
While their tasks are as similar as they are different, both Articles for Creation and New Page Review are urgently in need of additional, competent hands on deck. If you have the required level of experience and would like to spare some time helping not only to keep the backlogs down, but helping genuine but confused new users, please check out New Page Review and Articles for Creation and apply for the use of the tools.
Discuss this story