The Signpost

News and notes

Wiki Loves Monuments evaluation sees diminishing returns and increasing cost

Contribute  —  
Share this
By The ed17 and Resident Mario

The Wikimedia Foundation's first two program evaluations of 2015 have been published on Meta. These examine the annual Wiki Loves Monuments (WLM) and other photo competitions that have been held around the globe, with an eye towards finding what worked and what did not. Although WLM is an international contest, it is organized separately in each country, with separate budgets and contests in each before the winners advance to the global finals. It was first held in the Netherlands in 2010, and the success there encouraged organizers in other countries to join.

The evaluations reveal that in the last three years (2012–14), WLM has possibly fallen victim to its own success and seen diminishing returns: the average total number of uploads has decreased from a high of 6,266 images in 2012 to 2,714 in 2014 even as the average money spent per upload has increased from US37c to 90c. While the total number of images increased from 2012 to 2013, it was less than half that in 2014. The number of images in use on Wikimedia programs has dropped both in total number and percentage, and the cost per used image has gone up from $3.03 in 2012 to $6.31—although this is an improvement on 2013's $6.61. Cost per participant was, on average, about $25. About 1% of the total image uploads were later rated as "quality" or "valued" images.

The users participating skewed heavily towards people who had never edited Wikimedia sites: over the three studied years, about 1,400 current and 14,000 new users participated. The conversion rate into continuing editors, as measured by having at least one edit three months after the competition, is 2.4%. Extending this to twelve months after the competition (for programs that ended before February 2014) shows that the programs netted 16 "active" new editors, or 0.3%—those who made more than five or more edits in the studied period.

The overall data analysis by the WMF suggests that "When planning a photo event, it may be useful to try to balance group size with both new and experienced users to increase use and ensure high quality uploads." For funding, the evaluators recommended that the WMF be "cautious about the investment level" amidst the contest's diminishing returns. E

Brief notes

Foundation staffers modeling Wikipedia Store t-shirts. The online merchandise portal re-opened this week and is currently giving away free merch to nominated Wikimedians active on the projects.
Purchasable highlights include a free-knowledge t-shirt prominently featuring the Creative Commons CC-0 logo, literally plantable pencils ("who says knowledge can’t grow on trees?"), and your correspondent's personal favorite item, the "rabbit hole" t-shirt, featuring a visual depiction of the WikiWalk. In support of the relaunch, a merchandise giveaway allowing Wikimedians to nominate other users for free merch is currently also underway. R
+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.
  • Why is this article trying to monetize photos from participants. Gives me a horrible feeling, the community/participants are not a factory plant in what every employee needs to work a minimum number of hours. The goal of Wiki Loves Monuments is to get all monuments with a good picture on Wikipedia. The first time a contest as such is organised the low hanging fruits are done first, but in this article it is forgotten to mention that getting the low hanging fruits is not the core goal of Wiki Loves Monuments. The goal of Wiki Loves Monuments is to get a photo of every monument. The more monuments get a picture, it becomes much harder to ge a picture of the other monuments. Please, if you want to write a story about a subject, do not only write about too easy thoughts without thinking it through. This article is failing in describing the actual situation and misses totally what Wiki Loves Monuments is about. Romaine (talk) 04:20, 1 May 2015 (UTC)[reply]
@Romaine: This article is a simple summary of the WMF's evaluation. I would take up your concerns with them! In their defense, however, I would note that donors' funds are being used to pay for these uploads. While the cause is noble (and one I personally laud), the WMF needs to consider how much of an impact their funds are having in order to avoid waste. Ed [talk] [majestic titan] 05:24, 1 May 2015 (UTC)[reply]
Measurables are in the air. Surely we should be concerned with the costs of the programs that our readers' money is funding, no? That is what the analysis is all about. ResMar 05:32, 1 May 2015 (UTC)[reply]
I agree with all of you. I think cost analysis is relevant to justify donor money. However the above summary is too much focused on money. There are figures in the WMF report that point to the effectiveness of WLM in improving Wikipedia that could have been included, such as that only 13% of images are used in any Wiki project and that only 0.03% became featured. Together with the 0.3% editor retention it shows that WLM is still wasting energy by focusing on quantity rather than quality and usefulness. --ELEKHHT 09:36, 1 May 2015 (UTC)[reply]
Historical marker and access sign
But perhaps there is more to it than that.
  • The images are available for all time, for everyone. The cost of a commercial photograph starts at several hundred dollars.
  • There is awareness raising value from the events. For example if someone makes their flikr repository CCBYSA-4 there is a huge hidden gain.
  • Though we like "active editors" since they do "most of the work", there is no guarantee that this will remain the case. A million editors who make one good edit a month would be fantastic.
  • There are millions of minor monuments that should be recorded and aren't. For example the other side of the marker on the right here.
All the best: Rich Farmbrough12:17, 1 May 2015 (UTC).
If the article is simple a summary of WMF's evaluation, I think both fail in describing it correctly. Wiki Loves monuments is organised to have a full coverage of the worlds cultural heritage, not just of the most popular or easy monuments. It is a nice benefit that we broke the record of the world's largest photography competition, but that is not the core goal. The core goal is to have the world's heritage covered in pictures on Wikipedia, which means a cumulative grow in number of photos with each contest. The comparison of the budget compared to the number of uploads is also strange, like there is a strong relationship between them. The number of uploads depends on so many parameters that are outside the control of an organising team, and so many depends on just having luck. Yes, luck, that is an underestimated parameter with any contest and with many other things on Wikipedia as well. The only thing an organising team can do, is doing their best they can.
"I would note that donors' funds are being used to pay for these uploads" -> This is not true. None of the uploads is paid. The donors' funds are being used to organise a large photography in what thousands of volunteers participate with taking and uploading photos to help Wikipedia improve. But as you are playing the "donors"-ball, ask this question: do the donors want only the most popular monuments to be covered on Wikipedia, or all cultural heritage monuments? – I speak with many many people, including donors, and none of them expect us to cover only the most popular monuments. Everyone expects that Wikipedia has them all.
It is fine to evaluate a contest in comparison, but such should be done in comparison with other ways of getting the exact same results. The current set-up is too much focussed on the wrong goal. The goal is not spending less money. The goal is getting the best possible results for the money spend, to get the same results.
Also the report, and this article, totally exclude other side effects that Wiki Loves Monuments has. For example that the contest has resulted in a image donation of 480.000+ images of monuments.
"Together with the 0.3% editor retention it shows that WLM is still wasting energy by focusing on quantity rather than quality and usefulness." -> Sorry, this is nonsense. Like the organising teams have any choice in what kind of photos participants upload. And there is no focus on quantity. And there is no focus on editor retention. That WMF sets itself this goal, fine, but this is not the goal of Wiki Loves Monuments. Wiki Loves Monuments has as goal to have a better coverage of the world's heritage. If you want to compare Wiki Loves Monuments with what you would like, fine, but that is not the goal of Wiki Loves Monuments and comparing with other people's goals makes no sense. Romaine (talk) 07:01, 2 May 2015 (UTC)[reply]
  • Am I the only person that finds it bizarre that the Wikipedia Store, which sells many items presumably mostly bought by editors, was launched without any large notification on the actual wiki? Just an unofficial mention in the Signpost? Sometimes it feels like Wikimedia is off on its own planet, with whatever 1000 people apparently fly around the world each year to tiny Italian villages to discuss things WM is launching that year, and is vaguely surprised each time they are reminded about the several dozen thousand people that create their content every day. --PresN 04:26, 1 May 2015 (UTC)[reply]
The mailing list, which did indeed run notification, to the very tightest core of Wikimedian power users. You would be wrong in that assumption, that it sells to editors—it seems to me to be a most very literal gift shop, or pitched as such. Without seeing their traffic stats (perhaps I should?) I cannot say whether or not that is the right approach, but that is the approach. Also incongruous at least to me: in a time where we have long since moved away from Wikipedia-centrism, why "Wikipedia Store" and not "Wikimedia Store"? I think it has to do with name-brand recognition, which reinforces my notion that the gift shop is mostly literally that, for passerby. ResMar 05:33, 1 May 2015 (UTC)[reply]
Well, I would have gone with the global notification tool that puts a line on the top of people's watchlists, which is the general wiki-wide way to do it. Also: "The mailing list, which did indeed run notification, to the very tightest core of Wikimedian power users" -> "with whatever 1000 people apparently fly around the world each year to tiny Italian villages". If it's not aimed at editors, then whatever. --PresN 17:55, 1 May 2015 (UTC)[reply]



       

The Signpost · written by many · served by Sinepost V0.9 · 🄯 CC-BY-SA 4.0