Parser changes

New parser preprocessor to be introduced

On 24 January, a new preprocessor will be introduced that substantially changes the code that is used to parse many sorts of wikimarkup, in particular templates. This will fix a large number of bugs, and change the meaning of certain syntax. The message announcing this change is available here; a longer description is available at m:Migration to the new preprocessor, including a description of which of the changes are and are not negotiable.

The preprocessor is part of the parser that handles certain constructs that are handled before the other constructs on the page are parsed, including section edit links, transclusions, and comments. At present, all the pages run on the old preprocessor by default; but as "[a]ll known bugs in the new preprocessor have been fixed" it is now available as an alternative for testing purposes. It is known that some templates will cause problems with the new preprocessor (in particular, {{self}} has been singled out as a template known to have problems); therefore, the community's help has been requested to find uses of markup that differs between the two preprocessors and to fix it to be more 'correct'.

There are several categories of changes made by the new preprocessor; most of them fix bugs in one way or another, but some are due to differences between the way the two preprocessors operate:

One notable advantage to the new preprocessor is that branches of a parser function that are not used (such as the b in {{#if:1|a|b}}) are no longer expanded, resulting in lower pre-expand include values and reducing the length of time the server needs to parse such constructs.

Users are encouraged to test the new preprocessor before it goes live; there are two ways to do this. First, appending the option ?timtest=newpp (or &timtest=newpp if using /w/index.php rather than /wiki) to a page's URL will cause it to be rendered with the new rather than the old preprocessor. It is also possible to use Special:ParserDiffTest to check the difference between using the old and the new preprocessor on a page, using either subpage syntax or by entering some wikitext directly. Such testing will help to ensure that the change to the new preprocessor does not break large parts of the wiki when it finally goes live.

Also this week:
  • 2007 in review
  • Parser changes
  • Pic of the year
  • WikiWorld
  • News and notes
  • In the news
  • Features and admins
  • Technology report

  • Signpost archives

    + Add a comment

    Discuss this story

    These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.
    This seems pretty drastic. Is there a check-off list for what has and remains to be checked? MilesAgain (talk) 18:41, 22 January 2008 (UTC)[reply]
    Not as far as I know. The problem here is that there are some things that will be changed, and pages which can't handle the new changes need to be changed to allow for them. So the problem is to find which pages are using constructs that will become unsupported and correct them; I'm not sure if there's a list anywhere of pages where problems have been found but not corrected. --ais523 15:05, 24 January 2008 (UTC)
    Since going live on the 24th, there have been at least 7 new bugs found in the parser (two of which are still pending as if this post). The 5 fixed are:
    1. Comments can now share lines with editable section headers (per user request, as this method is widely used on wikipedia for noting section redirects).
    2. The msgnw: magic word was broken, fixed now.
    3. The first header in a template parameter or parserfunction was not editable. Two equals signs in a template or parserfunction now will take precidence as a possible editable section even over a template definition (although level 1 headers won't).
    4. Signatures in templates were expanding. Fixed.
    5. Occasional segfaulting on save.
    As for a to-do list of fixes, there are the two pending bugs. (permalink) as well as a list on m:Talk:Migration_to_the_new_preprocessor#Cleanup.2FTasks_.28TO_DO.29 of things that need fixing in wikicode. The only major bug left is the way $ parameters are being handled in messages. --Splarka (rant) 08:59, 30 January 2008 (UTC)[reply]


    The Signpost · written by many · served by Sinepost V0.9 · 🄯 CC-BY-SA 4.0