Translating SVGs and making history bugs history

Google Summer of Code: TranslateSvg

In the first of a series looking at this year's eight ongoing Google Summer of Code projects, the Signpost caught up with developer Harry Burt, which wasn't too tricky, given that he is also the regular writer of this report. Burt explained what his project was about, his success so far – final submissions are due in a month's time – and what impact it might have on the Wikimedia community:

The English-language version of a map of South Sudan created during 2011; alongside it on Wikimedia Commons are more than a dozen duplicates containing translated labels.

“

On 9 July 2011, South Sudan declared independence, and during that buzz, an Italian Wikimedian found his map showing the borders of the new nation had been translated into a dozen other languages, among them English, Greek, Catalan, and Macedonian. These copies were then uploaded onto Wikimedia Commons as separate files. Of course, one would expect the map to change significantly over the next decade. More often than not, these kinds of change are picked up first by editors of the larger projects, who rapidly update their own versions of the map. To do so takes, say, 20 minutes; but to replicate that same change across Catalan, Greek, Macedonian? Hours of work – and dozens of separate uploads.

My project, named "TranslateSvg", changes this workflow – for SVG format files at least – firstly by making it easier to translate those files (thus reducing the all-too-common sight of English-language diagrams in use on non-English wikis), and secondly by embedding the new translations within the same SVG file. When boundaries change, a single update will propagate to all language versions instantly. It's the smaller projects that will benefit the most, picking up those image updates that are performed every day by users of larger projects, but there are gains for larger wikis too from the reverse process.

I would say that progress has been good so far: the main hurdle remaining is code review, and it's during that period that the project will either sink or swim. If the latter, TranslateSvg could find its way onto Wikimedia Commons before the end of the year.

”

In brief

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

MediaWiki 1.20wmf8 begins deployment cycle: 1.20wmf8 – the eighth release to Wikimedia wikis from the 1.20 branch – was deployed to its first wikis on July 23 and will be deployed to all wikis by August 1. The release incorporates approximately two hundred changes to the MediaWiki software that powers Wikipedia, comprising 94 "core" changes plus a similar number of patches for WMF-deployed extensions. Among the changes (the product of a fortnight of development time, allbeit one disrupted by Wikimania) are improvements to the way Special:MyPage handles additional URL parameters (bug #35060; for example, &redirect=no) and a fix to Special:Contributions' "newbie" mode to exclude new bots from the display (revision #14674). A release to external sites including the same selection of bug fixes and new features is not expected for some time.

History bug marked resolved: After an investigation lasting nearly two months, developers are now confident they have fully resolved bug #37225. The bug, which intermittently caused null revisions to be erroneously recorded – and with non-zero byte change numbers – had prompted dozens of comments across many wikis, including at least 5 threads on the English Wikipedia's technical village pump alone. It was eventually traced to a small change in the exact SQL command being given to the database upon a page save, a problem which has since been repaired. The fix is included in the 1.20wmf8 release, but it is not known if it will also be applied (or indeed needs to be applied) retroactively to clear old revisions from the database.

Gerrit, but faster: The speed of Gerrit received a significant boost on Friday, when its databases were transferred from the Tampa, Florida datacentre to the WMF's other datacentre in Ashburn, Virginia, where the Gerrit frontend is hosted (wikitech-l mailing list). Developers have already noted a significant improvement in load times for all sorts of web-based Gerrit operations. The improvement comes in the midst of long discussions about the long-term viability of sticking with Gerrit, and more specifically this week, whether or not there was a "serious alternative" to the code review system, which has been in place since March (also wikitech-l).

Requested moves bot sidelined: RM Bot, the automated bot responsible for listing requested move discussions, made its last edit Wednesday. The PHP-run bot, which has performed since November 2009, is operated by recently inactive contributor HardBoiledEggs. The community is discussing returning to manually updating the listing page until HardBoiledEggs returns; in addition, current and potential bot-operators are advised that the existing and "indispensable" bot is currently in need of a new owner.

More dumps on the Internet Archive?: WMF developer and dumps guru Ariel Glenn this week blogged about her efforts to improve the quantity, quality and timeliness of WMF database exports (colloquially known as "dumps") hosted on the US-based Internet Archive. Wrote Glenn, "When we look back on this period of our digital history, the Archive will surely be recognized as one of the great repositories of knowledge, a project that changed forever the course of the Internet. Naturally we want Wikimedia XML dumps to be a part of this repository", before explaining her work on a new suite aimed at facilitating the upload process.

&action=info?: An old system for supplying human-readable metadata about a page could be reinvigorated if a formal Request for Comment (RFC) receives support from developers (wikitech-l mailing list). The &action=info system, which would complement existing pages such as &action=history, could eventually list dozens of pieces of information about a page, including such details as creation time, creator and number of revisions. Enabling the page would still require local community consensus; the current discussion relates to having such functionality in the code should projects then wish to use it.

New staff member: Long-time editor S ("yes his name really is just 'S'") Page has joined the Wikimedia Foundation as a software engineer in its Editor Engagement Experiments (E3) engineering team, WMF Director of Engineering Alolita Sharma announced this week (wikitech-l mailing list). A former ski instructor and sometime road sweeper, Page will help develop new technologies aimed at bringing in and then retaining new editors; it is not yet known what his precise focus might be.

One bot approved: 1 BRFA was recently approved for use on the English Wikipedia:

Hmainsbot1's 3rd BRfA, delinking date fields in articles per WP:DATELINK;

At the time of writing, 10 BRFAs are active. As per usual, community input is encouraged.

← Previous "Technology report"

Next "Technology report" →

In this issue

23 July 2012 (all comments)

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Thank for including details of the 1.20wmf8 rollout. If the history bug only affects the RecentChanges table, affected entries will anyway be deleted after 30 days (at least on enwiki), so it might be easier to let the past errors expire naturally instead of applying a retroactive fix to existing data. — Richardguk (talk) 21:44, 23 July 2012 (UTC)[reply]
History pages work on the revision table, not the recentchanges table. Bawolff (talk) 12:29, 24 July 2012 (UTC)[reply]
Bugzilla:37225 was identified because of corruption to article size entries on pages based on recentchanges (watchlists and RecentChanges). It did not seem to corrupt revision entries on history pages, though I'm not sure whether the possibility has been ruled out. "History bug" is a slightly misleading description in the Signpost article. — Richardguk (talk) 12:45, 24 July 2012 (UTC)[reply]
Oh ok, in that case I agree, retro-actively trying to fix RC table seems like a waste of effort. Bawolff (talk) 13:17, 24 July 2012 (UTC)[reply]
Yeah, this is mostly me getting confused, I think, probably because I forget that the watchlists pulls rc entries and not revisions. Apologies for any confusion (I've tweaked the prose slightly). - Jarry1250 ^{[Deliberation needed]} 13:46, 24 July 2012 (UTC)[reply]
Well the bug title itself says "history" in it. Bawolff (talk) 14:19, 24 July 2012 (UTC)[reply]
Aha! I'm off the hook *clutches gratefully to straws* :P - Jarry1250 ^{[Deliberation needed]} 14:24, 24 July 2012 (UTC)[reply]
I think you've earned yourself more than a few straws for your community feedback! — Richardguk (talk) 17:48, 24 July 2012 (UTC)[reply]
Enabling the page would still require local community consensus - As far as I understand, MzMcBride's proposal was for something enabled by default, not an optional feature. The reason why ?action=info is optional currently is because it has performance problems, the new action=info should it happen, would not have those problems (or have parts selectively disabled based on miser mode). Bawolff (talk) 12:29, 24 July 2012 (UTC)[reply]
p.s. For the curious, the old action=info looks like this: https://translatewiki.net/wiki/Main_Page?action=info Bawolff (talk) 12:32, 24 July 2012 (UTC)[reply]
Enabled by default in MediaWiki, maybe, but having it enabled on existing Wikimedia wikis would either require consensus or a pitched battle. - Jarry1250 ^{[Deliberation needed]} 13:46, 24 July 2012 (UTC)[reply]
or people could just convinently forget to re-disable it ;) On a serious note though, usually concensus isn't required for newly developed features (Arguably its an old feature that would be re-developed, but the only reason the old feature was disabled is to prevent the servers from coming to a crashing halt) As far as I am aware WM hasn't actively disabled it, wmf wikis are just using the current mediawiki defaults. Bawolff (talk) 14:19, 24 July 2012 (UTC)[reply]
I think we're seeing a sea-change on the view that "consensus isn't required for newly developed features", but perhaps it's more a question of whether any links to the new page are actually visible. - Jarry1250 ^{[Deliberation needed]} 14:24, 24 July 2012 (UTC)[reply]
Its what happens when people keep developing somewhat odd (relative to the wiki-way) and highly user-facing features. Although most of the controversial ones have been extensions which traditionally did require concensuss. Bawolff (talk) 14:31, 24 July 2012 (UTC)[reply]
Seems to me that, now that MzMcBride has refined the RFC to adding metadata at ?action=info (instead of moving metadata from edit/editpreview pages to ?action=info), support for the additional parameter should be largely uncontroversial and most editors would probably never even realise that it exists. But I am surprised that there was not more initial concern at the subproposal to disclose the number of watchers for any page, since moderately sophisticated vandals could learn to target pages that are not much watchlisted. — Richardguk (talk) 17:46, 24 July 2012 (UTC)[reply]
Honestly the chance that a feature would get accepted that disclosed the number of watchers a page has when there's already a right restricting that (unwatchedpages) is pretty low in all probability. Bawolff (talk) 19:48, 24 July 2012 (UTC)[reply]

To clarify, S Page is part of the experiments team at the WMF. :) Steven Walling (WMF) • talk 16:50, 24 July 2012 (UTC)[reply]

A critical thing to help SVG translation support would be to end the situation where people are driven to convert so much text into path data -- so much more difficult to edit and update, never mind search -- because the text handling and placement of WP's SVG renderer is so appalling. SVG has so much to offer, it's a crying shame that WP's SVG support is so poor, that people are forced to go to workarounds like this. The generally miserable state of libsvg has been an unaddressed issue for years -- will resources ever be put into either bringing it up to the mark, or replacing it with one of the other open-source SVG back-ends, that actually work? Jheald (talk) 11:04, 30 July 2012 (UTC)[reply]
- librsvg isn't as bad as it used to be, but agreed that flowRoot support would be a great addition. I think recent calls to reinvestigate used Inkscape to convert to PNGs would be a good idea. - Jarry1250 ^{[Deliberation needed]} 11:31, 30 July 2012 (UTC)[reply]