Last week, major search engines and makers of blogging software announced a new effort to fight link spam, fueling debate over whether to incorporate it into the MediaWiki software and implement it on Wikipedia. A feature to do this was quickly developed, and discussion turned to the possibility of selective implementation.

The three largest search portals, Google, Yahoo!, and MSN, announced Tuesday that they would support systematic use of a "nofollow" tag as a way to deter link spam. In conjunction with this, a number of major blog software makers indicated they would participate in the effort.

Link spam, sometimes also called "comment spam" because it is frequently placed in the comments section of blogs, has become a common but widely disliked technique in search engine optimization. The spammer places links to a website on many other sites that are open to editing, notably blogs that allow public comments as well as publicly editable wikis. Since search engines rely heavily on links between sites in ranking websites, the effect is to artificially raise the rank of the spammer's website in search engine results.

The "nofollow" tag serves as an instruction for the search engine's spider as it browses the links on a webpage. Google indicated that effective immediately, hyperlinks with this tag "won't get any credit when we rank Web sites in our search results." Yahoo stated that it would support this technique within weeks, and MSN plans to recognize the tag when it switches over to its Microsoft-developed search technology later this year.

According to a Google statement (on a blog, naturally enough), joining in the initiative are a number of notable providers of blog software and related services. These include Six Apart, WordPress, Blosxom, and blojsom, as well as the in-house blogging services of the search engines such as MSN Spaces and Google's Blogger. Also participating are photo-sharing services Flickr and Buzznet. Six Apart also announced its plans for implementing the tag on its individual blog services, including Movable Type and the recently acquired LiveJournal.

On Wednesday, following the public announcement of this initiative, Shizhao asked on the wikitech mailing list whether the MediaWiki developers were planning to join the major search engines in fighting link spam. (In its statement, Google indicated the idea could apply to any software that allows outside people to add links on a site.)

Tim Starling argued that the new feature needed to be enabled by default on MediaWiki installations, because the initiative "will only work well if everyone participates, otherwise spammers will just continue to spam every wiki because they couldn't be bothered checking which ones are using this feature." Brion Vibber agreed that this would be the case if the MediaWiki development team decides to support the campaign.

However, concerns were raised about whether implementing the tag on Wikipedia was desirable. Decumanus commented, "My experience is that spam gets cleaned up very quickly. Conversely, I find pleasure in adding legitimate external links, knowing it will help raise the page rank of those sites." Jimbo Wales expressed the opinion that the feature should be on by default in the MediaWiki software, but that Wikipedia itself should turn this feature off.

As Wikipedia has grown in popularity, it has become a more valuable target for spammers, but watchful editors have helped keep the problem under control. Individual spam links can be removed, and more concerted spamming efforts fought off with additional technical help.

The developers have already implemented several features to discourage spam. Known spammers have been placed on a spam blacklist that prevents edits linking to their websites from being saved.

Restrictions on bot editing also serve partially to deter bulk spamming efforts. According to the spam policy, bots that add external link spam can be treated as vandalism.

These measures have generally succeeded in keeping external link spam from staying on Wikipedia for any significant length of time. However, Tim Starling pointed out that some Wikipedia mirrors probably would not use the "nofollow" tag, meaning that spammers might still see some benefit from adding links to Wikipedia.

Also, spam has been a serious problem on other Wikipedias, particularly those in languages that have little activity and are difficult to monitor. The developers emphasized that the nofollow tag was especially suited for wikis that are largely unattended and more vulnerable to link spam.

As a result of their discussion, the developers enabled this use of the nofollow tag in the MediaWiki software and implemented the change. However, more objections had been raised in a discussion on the village pump, and ideas for selective implementation of the change had not yet been designed.

This led to further discussion and a poll on the Meta site about the issue. The poll started with only two options, either keeping the nofollow tag or getting rid of it. However, a third option, "Keep nofollow on unattended wikis, think about alternatives for active projects", was soon added and quickly became the preferred option of most participants.