English Wikipedia's peer review process is a relatively unknown yet fairly active venue for feedback. Editors can request a review of an article, regardless of its stage of development, in general form or highlighting specific areas they are looking to improve. One or more editors may weigh in.
Here, I outline the history of peer review and my contributions to it, and detail future plans and a request for more reviewers.
Peer review runs similarly to other processes such as good articles and featured articles: an editor may post a request, and one or more editors answer that request, critiquing the article. Editors who submit their work for review range from novices with their first article to those seeking feedback before nomination to the good article or featured article forums.
The peer-review venue was created in 2003 by Wapcaplet (now inactive); the first article reviewed was States of the United States. The process quickly gained popularity. Reviews were manually added to a list, until a 2005 redesign based on transclusion. Editors had to manually add and remove articles until 2008, when the processes for adding articles to the list and for closing old reviews were automated by VeblenBot (inactive), created by CBM (inactive), and reviews were split into categories.
After creation, peer review rapidly developed a set of instructions and advice. A "policy" page formalising review closure criteria had developed by 2005, created first by Bishonen. Ta bu shi da yu (inactive) began to archive old reviews from 2005. A list of volunteers by topic area was created in 2007 by Marskell (inactive). From 2008, Ruhrfisch began a backlog of unanswered reviews. In 2012 an "alert" template cautioning editors from directly posting reviews on the main peer review page was created by Ruhrfisch.
I first encountered peer review in 2013. In comparison to processes such as "good articles" and "featured articles" I found it messy and disorganised. These had developed friendlier interfaces, clearer instructions, had split into several pages for ease of use and load times (a crippling issue for me, using wireless internet in a rural area at the time), had a greater degree of automation, and more active users. These had changed with time; peer review appeared mostly unaltered for many years. Three things struck me:
Finding the actual peer reviews very useful, I have since striven to gradually improve the process, with discussion and consensus. My goal has been to make it easier to use, and therefore more widely used, as part of Wikipedia's growing infrastructure for supporting new editors, and another way of helping users non-confrontationally gather feedback about their articles.
I started by creating a tabbed header of commonly used pages, similar to good and featured articles; and by branding the peer review pages with a colour scheme. I created a sidebar with other main pages (identified via "what links here" and also "subpages") and put links to as many relevant templates into the "tools" subpage, where I tried to document the byzantine template- and bot-based technical underpinnings of the peer review process.
A sample:
The Category Format (CF) templates are a suite of templates to be used in conjunction with automatic category listing. An automatically listed category is a category which has an associated category list page where the items in the category are listed ... for maximum flexibility, the entries on the category list page are formatted using a subtemplate of CF named after the category. For instance each page in an automatically listed Category:foo has a corresponding entry.
I have centralised information, truncated text, simplified instructions, and made small changes to ease reviewing (such as including the date of creation and last edit on all displayed reviews; slipping in a request for reciprocity in instructions). I have simplified the technical side of things for future editors (this has included creating simpler and comprehensive documentation; centralising the 11 identical templates; simplifying the archive process.) We now link to categories of monthly reviews; previously, an editor had to copy and parse reviews each month, a time-consuming process that understandably was often months behind.
Reviews would not transclude to the main page during mid-2015, due to a technical cap on the amount of text that can be transcluded. When this is exceeded, nothing transcludes and, with great foresight, peer review receives a message from VeblenBot. With too many reviews (and delays in automatic closure), this limit had been exceeded. We quickly implemented a fix – reviews of a certain length now display their text on the main page. After this limit, a link is provided to the main review. I hope in the future that we continue to have this problem, because it indicates that the process is being used more and more.
We've had long-running issues with on-again–off-again bots over the last two to three years, which has impacted the displaying of new reviews, and the closing of old reviews. I hope this has been finally solved by AnomieBOT (thank you!!). The bot was designed in response to requests at the technical village pump and bot requests, and even a failed attempt I made to learn bot coding. This again illustrates the frustrating aspect of a small bus factor, particularly for such a critical bot as VeblenBot.
The "unanswered reviews" list was automated in 2016. This replaced a "backlog" template that was infrequently edited, with months often lapsing between edits due to the complexity and time-consuming nature of the task. An editor has to manually scan the list of reviews (daily if the template were to be kept up to date) and add reviews older than a certain duration to the template; and check all reviews on the template to see if they had been reviews. This required a lot of effort and months often passed without reviews. The current automatic method is imperfect and not entirely satisfactory, but at least provides a more up-to-date list. It displays only reviews that have never been edited. I should mention and thank on behalf of the community Cirt (inactive) and Brianboulton (active), who maintained the backlog list for some time.
Peer review is busier and busier than ever, and is attracting more and more reviews. This is great news and I'm pleased to see the diverse nature of requests posted. But we're still plagued by a lack of reviewers; there's nothing special needed to become a reviewer, and frankly not that much time needed for a review.
Most editors, in my experience, are seeking some general tips about the articles they are editing – general help about sourcing, structure, titles, images and so forth. Interested editors have the potential to help ease newer editors' introduction to Wikipedia – this may appeal particularly to those civic-minded people who help out at the teahouse.
Editors seeking review often do so for no other reason than a desire to improve articles and better write articles in the future. By helping these editors, who are often new, you can make them feel welcome, and make a positive contribution to their future edits and to the encyclopedia as a whole. I hope I've given you an idea of the evolving nature of peer review, and that you'll consider participating! Please drop by and have a look at peer review every so often, and pick up one or two reviews.
Discuss this story
Peer review is something I need to do more of. As a comment, what would be really good would be setting up some kind of "expert peer review" (maybe as part of review for an FA submission?) in which actual, credentialed experts are invited in to give a public assessment of articles. I've heard many times from experts that they grumble about how Wikipedia is bad - not just factual mistakes, that an article overemphasises the wrong things, or spends too much time talking about a theory most experts on the topic think is on the way out, or the hot new controversial idea that isn't very plausible. (See, for example, Emily Gould's comments on her article.)
It would be nice if we could actually invite people who know about a topic to publicly say this stuff and use it as a guideline for improving an article. I often find that experts on a topic don't automatically make the best writers about it - they may be too embedded in a point of view to give due weight to different sides of a debate - but having this kind of opinion on the talk page would be nice. Finding actual qualified experts like academics - or publicly known people like journalists, political consultants or lawyers - willing to donate their time to this kind of thing is the kind of thing the WMF sounds like it would be well-placed to negotiate (it would probably be a more efficient use of their time than actually editing the article directly and figuring out how the formatting works). How about it, guys? Blythwood (talk) 03:53, 28 February 2017 (UTC)[reply]
Did you mean for this to be in Category:X1?
@LT910001: in an early edit, you put in an example. A bot changed it from foo to X1. Did you mean to put this page into that category, or did you mean to type , which would have rendered as without adding the page to any category? davidwr/(talk)/(contribs) 22:40, 18 March 2017 (UTC)[reply]