The Signpost

Technology report

MediaWiki 1.20 and the prospects for getting 1.21 code reviewed promptly

Contribute  —  
Share this
By Jarry1250

Code review statistics bounce back in October after difficult September

In late September, the Technology report published its findings about (particularly median) code review times. To the 23,900 changesets analysed the first time (the data for which has been updated), the Signpost added data from the 9,000 or so changesets contributed between September 17 and November 9 to a total of 93,000 reviews across 45,000 patchsets. Bots and self-reviews were also discarded, but reviews made by a different user in the form of a superseding patch were retained. Finally, users were categorised by hand according to whether they would be best regarded as staff or volunteers. The new analyses were consistent with the predictions of the previous analysis.

Our investigation found that September represented a particularly poor month for code review (across both extensions and "core" MediaWiki code) but that this loss was more than picked up in October, which was the best month on record for code review. Specifically, 50% of patchsets submitted during October were reviewed just two and a half hours after submission, and 75% within 18 hours. The 95% percentile remains stubbornly high at nearly two weeks, suggesting that finding reviewers for certain types of patch remains hard.

The staff–volunteer divide highlighted in the last report remains. The median patchset was reviewed twice as quickly if you were a staff member working on an extension in October rather than a volunteer, and although it is too early to tell conclusively, there seems to be a similar gap for contributors to "core" and/or WMF-deployed extensions. 44% of all-time first reviews come from five reviewers (all staff), though this figure is down from 55% at the time of the last report, suggesting a significant diversification in the last 7 weeks. On a positive note, the percentage of all-time first reviews coming from volunteers has also increased – from 14% to 25% – as the Foundation gives a large number of volunteers more reviewing power.

As with any statistics, these figures should be taken with a degree of caution. The full dataset is available upon request.

In brief

Signpost poll
In my view, videos are... (a) ...integral to future success (37%) (b) ...nice but not vital (56%) (c) ...distractions (7%)
You can now give your opinion on next week's poll: This week saw a discussion about the difference between assigning a bug to the "lowest" priority setting and suggesting it will never be fixed. Which of these best sums up your response: Dealing with bug requests is a case of...?

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

At the time of writing, 18 BRfAs are active. As usual, community input is encouraged.
+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Next week's poll

Next week's poll is extremely subjective. All of the available answers (except "Other") rely on the assumption that lowest-priority bugs are worthless. I (and many other) strongly disagree. Lowest means in particular "Patches very welcome", and it is often a gathering point for motivated volunteers to write a patch and get an ACTUAL PROBLEM FIXED. Quite the contrary of "worthless". Please cancel this poll, as its results will necessarily be flawed and mis-used. Also: You write "This week saw a discussion [...]", so the least you could do is provide a link to said discussion. Previous polls were usually of high quality, so I am surprised by how obviously unethical this one is. Nicolas1981 (talk) 03:25, 15 November 2012 (UTC)[reply]

The poll reads:
This week saw a discussion about the difference between giving something the "lowest" priority and suggesting it will never be fixed. Which of these best sums up your response: Dealing with bug requests is a case of...
...telling the truth, the whole truth and nothing but the truth, however brutal
...omitting "unhelpful" relevant facts, but never actually lying
...telling the odd white lie if it encourages future participation/engagement
Other / None of the above
Wow. That is just about the worst wording for a poll I have ever seen. It assumes that bugs are worked on in order of priority. That isn't even true where you pay the developers and it certainly isn't true when you have volunteers. Sometimes a lower priority bug is a very simple fix while the higher priority bug has you stumped. Sometimes you simply don't agree with whoever assigned the priorities. Sometimes someone else is working on the high priority task and you don't want to step on his toes.
But does the poll take any of this into account? Nope. It just assumes bad faith and decides that anyone who assigns a bug to the lowest priority must be a scheming liar and that only those who assign it a don't fix priority are honest and truthful.
I happen to like "won't fix" categories, but I use them to indicate bugs where even if you fix the bug your fix will not be accepted. For an example, look at the very first patent at List of Edison patents. That one is a classic "don't fix". See for details. In my opinion, this poll is POV pushing, plain and simple. --Guy Macon (talk) 08:31, 15 November 2012 (UTC)[reply]
While the Signpost's writing is generally quite good, several polls lately have been problematic. Perhaps you should run them by someone (perhaps a non-Signposter) for a) clarity, b) completeness, and c) objectivity before publishing them? --Philosopher Let us reason together. 16:19, 15 November 2012 (UTC)[reply]
While there is a subset of bug reports that end up as "lowest" priority or WONTFIX (my guess is less than 10%), I don't see the relation to "Dealing with bug requests is a case of" which refers to ALL bug reports. When it comes to the available answers, what is "telling the odd white lie" meant to describe for a valid and normal or even critical bug report? There simply is nothing to lie that my mind can come up with. "Thanks for reporting, valid issue, should be fixed, patches will speed it up", but where and how to lie here in a way that "encourages future participation"? I don't get it, or my imagination doesn't work well on Friday mornings. --Malyacko (talk) 09:58, 16 November 2012 (UTC)[reply]
Some of these responses seem a bit harsh about the poll, but I do also tend to agree with you that lying does not encourage future participation. Being honest causes future participation because then people's time is not wasted. I could perhaps somewhat understand not wanting to mark a wontfix bug as wontfix simply because in some controversial issues the users will yell at you (StringFunctions anyone?), but that's not an excuse for not marking bugs how they need to be marked. Additionally I'm not sure where said discussion took place [but then again I haven't been following all technical discussion recently. There was one about RESOLVED LATER, but that's a tad different]. Bawolff (talk) 00:18, 17 November 2012 (UTC)[reply]
The second part of the question doesn't need to be related to the first and is not. The fist is only needed to pretend that the poll has some relation to news; the relation being that those discussions and changes had as a stated objective to communicate more clearly/truthfully the status of bugs. The proposed answers were not really problematic because they didn't affect the choice or understanding of what to do next (unless one used a lot of imagination), but they surely were not about what I consider to be the "dealing with bug requests" and I voted "Other" to represent it.[1] [2] [3] --Nemo 08:20, 27 November 2012 (UTC)[reply]
  • Afterword: In the 2012-11-19/Technology report, Jarry1250 announced: "In response to apt criticism of last week's poll title, I have decided to retire the polling feature until I can devote sufficient time to administering it properly." I'd like to take the opportunity to thank Jarry and the Signpost team for their ongoing hard work producing fresh and engaging reports each week and helping to bridge the gap between editors and techies. — Richardguk (talk) 06:00, 22 November 2012 (UTC)[reply]


The Signpost · written by many · served by Sinepost V0.9 · 🄯 CC-BY-SA 4.0