The Signpost

Recent research

Lessons from the research literature on open collaboration; clicks on featured articles; credibility heuristics

Contribute  —  
Share this
By Taha Yasseri, Piotr Konieczny, Aaron Shaw, Tilman Bayer and Lui8E

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Lessons from the wiki research literature in "American Behavioral Scientist" special issue

A special issue of the American Behavioral Scientist is devoted to "open collaboration".

A fourth paper in this special issue, titled "The Rise and Decline of an Open Collaboration System: How Wikipedia’s Reaction to Popularity Is Causing Its Decline", found considerable media attention this month, starting with an article in USA Today. It was already reviewed in the September issue of the research report.

Mathematical model for attention to the promoted Wikipedia articles

While the size and growth rate, editorial workflow, and topical coverage of Wikipedia have been vastly studied, there is little work done on the understanding of public attention to the Wikipedia articles. In a working paper by a team from the Barcelona Media Foundation and the University of Twente, placed on arXiv just before Christmas,[4] the number of clicks on the featured articles promoted to the Wikipedia Main page is analysed and modeled.

A total of 684 featured articles are considered and the page view statistics of them is rescaled by the average circadian view rate extracted from a larger set of 871 395 articles in a period of 844 days. The 4-day lifetime of the promoted articles on the Main page is characterised by four phases. A very rapid growth in the number of article clicks just after the article appears on the Main page, followed by a rather homogeneous period of the first day of the promotion. As the article is replaced by a new featured article, and placed in the "recently featured" part of the Main page, the rate of clicks drops dramatically, and finally, the fourth flat phase is experienced during the remaining 3 days at this location.

In the next step, the authors introduce a rather intuitive model based on a few parameters to fully describe the 4 days cycle in a mathematical framework. The model is tuned based on the data of a set of 100 featured articles to predict the number of page hits for the rest of the sample, given the number of the clicks after the first hour of promotion for each article. The model is relatively accurate in predicting the number of clicks, and this accuracy could be even improved by feeding the model with the number of clicks at the end of the first day instead of the first hour after promotion. While the paper is very clear in describing the methodology, it fails to discuss and provide a deeper understanding of the social mechanisms of popularity and public attention, as it is mentioned repeatedly by the authors.

The featured article icon and other heuristics for students to judge article credibility

A paper in Information Processing and Management titled "College students’ credibility judgments and heuristics concerning Wikipedia"[5] used the theory of bounded rationality and a heuristic-systematic model to analyze American college students’ credibility judgments and heuristics concerning Wikipedia. Not surprisingly, authors observe that students used a heuristic (a mental shortcut, such as An article with a long list of references is more credible than with of a short one) in assessing the credibility of Wikipedia. Students (regardless of their knowledge) were much more likely to focus on the number of references than on their quality, and the same article would be seen as more credible depending on how many references it had. The authors conclude that educators need to teach students how to judge the quality of Wikipedia articles that goes beyond checking whether the article has references (and how many). The authors recommend that Wikipedia makes its own assessments (such as the Featured Article star, currently visible only as a small bronze star icon on the top right-hand corner of the article’s page) much more prominent. (This reviewer strongly agrees with the conclusion, but unfortunately the last community discussion appears to have achieved little.)

More interestingly, the authors also find that people with more knowledge found Wikipedia more credible, suggesting that people with low knowledge may be more uneasy with Wikipedia. The authors suggest that the reliability of Wikipedia would be increased if more professional associations implemented programs such as Association for Psychological Science Wikipedia Initiative. In addition to getting the experts more involved in Wikipedia content creation, the authors suggest that a good idea may be for "professional associations themselves [to] provide their own endorsement for the quality of articles in their fields."

The authors also note that peer endorsement is an important factor in credibility, and that the Wikipedia:Article Feedback Tool is a step in the right direction, as it provided another credibility assessment for the readers. They note, however, that compared to similar tools implemented on other sites (such as Amazon), "Wikipedia readers need to click on ‘‘View Page Rating,’’ which requires one more step to find out that information. The average reader may not be inclined to do so. It would be useful to display ratings without clicking".



  1. ^ Andrea Forte, Cliff Lampe: Defining, Understanding, and Supporting Open Collaboration: Lessons From the Literature doi:10.1177/0002764212469362 American Behavioral Scientist January 11, 2013
  2. ^ Elisabeth Joyce, Jacqueline C. Pike, Brian S. Butler:Rules and Roles vs. Consensus: Self-Governed Deliberative Mass Collaboration Bureaucracies doi:10.1177/0002764212469366
  3. ^ Brian Keegan, Darren Gergle and Noshir Contractor: Hot Off the Wiki: Structures and Dynamics of Wikipedia's Coverage of Breaking News Events. American Behavioral Scientist. doi:10.1177/0002764212469367 PDF
  4. ^ Marijn ten Thij, Yana Volkovich, David Laniado, Andreas Kaltenbrunner: Modeling and predicting page-view dynamics on Wikipedia PDF Open access icon
  5. ^ Sook Lim: "College students’ credibility judgments and heuristics concerning Wikipedia". Information Processing & Management, Volume 49, Issue 2, March 2013, Pages 405–419 doi:10.1016/j.ipm.2012.10.004 Open access icon
  6. ^ Reagle, Joseph (January 2013). ""Free as in sexist?" Free culture and the gender gap". First Monday. 18 (1–7). Retrieved 2013-01-31. Open access icon
  7. ^ Ford, Heather (2013-01-14). "Why Wikipedia is no 'proxy for culture' (Part 1 of 3)". Ethnography Matters. Retrieved 2013-01-31. Open access icon
  8. ^ Kevin Crowston, Nicolas Jullien, and Felipe Ortega : Sustainability of Open Collaborative Communities: Analyzing Recruitment Efficiency Open access icon
  9. ^ Fujiwara, Yuya and Konishi, Yukio and Suzuki, Yu and Nadamoto, Akiyo: "Extracting lack of information on Wikipedia by comparing multilingual articles"Proceedings of the 14th International Conference on Information Integration and Web-based Applications & Services doi:10.1145/2428736.2428808
  10. ^ Finn Årup Nielsen: "Sentiment analysis of Wikipedia pages on Danish politicians". Finn Årup Nielsen's blog. 2013-01-09. Retrieved 2013-01-31. , "More on automated sentiment analysis of Danish politicians on Wikipedia". Finn Årup Nielsen's blog. 2013-01-09. Retrieved 2013-01-31.
  11. ^ Nguyen, Thanh Hoang and Nguyen, Huong Dieu and Moreira, Viviane and Freire, Juliana: Clustering Wikipedia infoboxes to discover their types. Proceedings of the 21st ACM international conference on Information and knowledge management, CIKM '12 doi:10.1145/2396761.2398588 Closed access icon
  12. ^ Shiju Alex: Analysis of the Indic Language Wikipedia Statistical Report 2012
  13. ^ Brett Kuprel. Detecting Bad Wikipedia Edits. PDF; Mudit Jain, Murugan Ayyappan, Nikhil Agarwal. Detecting vandalisms in Wikipedia edits. PDF; Tony Jin, Lynnelle Ye, Hanzhi Zhu. Detecting Wikipedia Vandalism. PDF
  14. ^ Leonhard Dobusch: Sighted Versions in Wikipedia: A Case of Algorithmic Governance
  15. ^ Olivier H. Beauchesne: A Map of the Geographical Structure of Wikipedia Links
  16. ^ Olivier H. Beauchesne: A Map of the Geographic Structure of Wikipedia Topics
  17. ^ Andreas Kaltenbrunner, Pablo Aragón, David Laniado, Yana Volkovich: Not all paths lead to Rome: Analysing the network of sister cities
  18. ^ Erik Zachte: Monthly page requests, new archives and reports
  19. ^ David Ludwig: "Extended Cognition in Science Communication"
  20. ^ John Wallis: "Destructive Editing and Habitus in the Imaginative Construction of Wikipedia"
  21. ^ Joseph Reagle: The new scholasticism
+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.
  • I'd broadly agree with the idea of making ratings more visible, but I think this really has to come after a rethink and an update of the way we handle quality-rating metadata. At the moment, talkpage assessments are frequently several years out of date, and often patchily assigned. There's been some work on resolving this, but ultimately it's going to involve a lot of heavy labour and a lot of volunteer hours. Andrew Gray (talk) 17:27, 31 January 2013 (UTC)[reply]


The Signpost · written by many · served by Sinepost V0.9 · 🄯 CC-BY-SA 4.0