The Signpost

Special report

A post-mortem on the Indian Education Program pilot

Contribute  —  
Share this
By Skomorokh

Synopsis

Initiated in the wake of last year's successful Public Policy Initiative, the India Education Program is the latest culmination of the Wikimedia Foundation's stated strategic goals of university outreach and expansion into the Global South. It quickly ran into difficulty, however, when it emerged that a significant proportion of the articles submitted by students in the program failed to show an adequate knowledge of basic editing skills and did not respect Wikipedia's copyright and plagiarism policies. Many students seemed manifestly unfamiliar with rudimentary wiki syntax and fundamental competencies such as sandbox creation and responding to talk page messages, and a minority appeared to lack the English language skills requisite for productive interaction. Furthermore, many of the topics assigned for editing by students were in technical areas traditionally well-served by Wikipedia's native editing community, which made for a difficult time for comparatively inexperienced students trying to improve advanced content. These issues were exacerbated by the impositions of regular academic deadlines that put pressure on students to submit something for fear of failing classes.

Frank Schulenburg, Global Education Program Director for the Wikimedia Foundation, who described himself as "deeply frustrated" with the outcome of the program.

As the talkpage of the project shows, the resulting cleanup efforts, including a contributor copyright investigation (CCI), generated much dissatisfaction and questioning of the preparations and management on the Wikimedia Foundation's end. Disgruntled administrators and new page patrollers accused the program leadership of failing to anticipate and pre-empt the enormity of the maintenance task facing community volunteers, and the entrenched cultural differences in understanding of plagiarism between the editing community and the students. They also questioned how it was appropriate for what was ostensibly a pilot program to involve more than eight hundred students. In response, Nitika Tandon, the consultant directly overseeing the project, acknowledged the copyright concerns and outlined the steps that had been taken to address them, including in-class lessons on copyright, requiring students to submit their work in sandboxes prior to ambassador or professorial approval, and one-on-one counselling for offending students. IRC office hours with the IEP team were held on October 12 and October 21. The copyright violations continued unabated however, and some editors went so far as to question why the community was tolerating such an initiative, even suggesting the program should be brought before the Arbitration Committee prior to any future activity.

On October 18, in the light of the continuing onslaught of copyright violations, arbitrator emeritus Wizardman called for the project to be unilaterally shut down, citing it as a net negative for the encyclopaedia. In the ensuing discussion, Kudpung, who had been leading the cleanup efforts, highlighted two unwelcome impacts upon the Wikimedia movement he called "blatantly obvious" but which seemed "to have been totally disregarded" – that students would not be attracted to edit Wikipedia by being put through a stress-inducing deadline-tied program led by highly inexperienced ambassadors, and that the community's administrators and new page patrollers had been demoralised by having been forced to deal with cleanup of problems not of its making. After a further week of problems, Calliopejen1 proposed a wholesale removal of all unsourced text contributed for the project. On November 3, following a meeting with the Director at Pune's College of Engineering, one of the participating institutions, Tandon announced the decision to call for an immediate end to the students' editing of Wikipedia, and for a one-month moratorium on student contributions while the backlog of copyright investigations continued. Students who had added good quality content would be rewarded with marks based on the quality of their edits, those who had either added plagiarized material or had not even started would lose marks, and anyone who continued editing would receive negative marks. As 13 out of 14 classes at the Symbiosis College of Economics had already concluded, only two classes remained active in the program at the time; one each at Symbiosis School of Economics and SNDT Women's University.

Apologising for the massive cleanup effort presented by the initiative, Wikimedia Foundation's Global Education Program Director Frank Schulenburg expressed 'deep frustration' with the outcome of the program. He admitted fault on the organisers' part in the delay of getting online ambassadors working with the students, an inadequate articulation of the ambassadors' role, and poor communication on the foundation's part, but tried to dispel the notion that any particular group was primarily culpable for the difficulties. Copyright specialist administrator Moonriddengirl, who is the current community liaison for the Wikimedia Foundation, expressed sympathy with beleaguered volunteers exasperated by the perceived lack of assistance from the foundation, but emphasised that staff were handcuffed from intervening officially by the foundation's Section 230-determined legal imperatives not to act as a publisher of content. Nitika Tandon has prepared a draft summary of "findings and learnings" from the program.

The administrator's perspective

The New Page Patrol backlog for the month from October 4 to November 4, cited by Kudpung as illustrating the herculean efforts of volunteer patrollers in addressing the flood of submissions.

The Signpost asked admin Kudpung for his reflections on the experience of dealing with the administrative headache generated by the program:

"My involvement began when I blocked the IP address of one of the faculties in an attempt to stem the massive flow of copyright violations. It was only after my curiosity took me deeper into the issues, that I became fully aware of the scale of the problem. My talk page became for a while the hub of communication – a situation that should never have arisen, but it was very difficult to know who was in charge of various parts of the project and whom to address. I've been teaching here in Asia for many years, and I was also concerned that during the planning stages the American side of the operation may not have taken the challenges of the cultural dichotomy into consideration. I spoke with several members of the US WMF staff in an endeavour to learn if they were aware of the extent of the issues and if anything of consequence was being done. I also spoke with Hisham several times to obtain some reassurance on behalf of the community that something would be undertaken at ground zero, and finally the organisers held discussions in the USA. Some of the things I posted on various talk pages may have been perceived by some as accusatory in tone, but I felt it was necessary that people be galvanised into action."

"I understand the importance of Wikipedia reaching out to other regions, especially to those like India that have strong ties to the English language, but the lesson drawn from this pilot project is that things work very differently in other countries, and careful, long-term preparation with the involvement of the community is essential."

"For future extensions of the project, it is paramount that the Indian Campus Ambassadors are more accurately selected and trained, and have an adequate working knowledge of editing and basic policies. It is equally imperative that new editors can benefit from user friendly page creation tools, and that a much improved system for the control of new pages along with a replacement for CorenSearchBot are made available to page patrollers as soon as possible – the next wave of Indian students and their ambassadors is going to need them. Ironically, if WP:ACTRIAL [Ed. A local initiative to restrict page creation to autoconfirmed editors that the Foundation chose not to implement] had been implemented, it would have spared the students much of their disappointment and embarrassment, and the maintenance community much of the stress they volunteered to be subjected to. Nevertheless, from what I have seen of the new tools already, I am optimistic that the WMF's core philosophies and outreach programs can be further developed and maintained."

The consultant's view

Hisham Mundol, a consultant for the Wikimedia Foundation with responsibility for the India Programs, on day 2 of the Campus Ambassador training in Pune.

The Signpost interviewed Hisham Mundol, consultant for the Wikimedia Foundation's India Programs, to get an insight into his perspective on the situation and what it might mean for the future of such initiatives. A transcript of the interview follows.

The Signpost: What was the planning process behind the India Education Program, what was the involvement of the different roles (e.g. contractors, foundation staff with prior experience with university outreach, foundation management, Wikipedians, professors, ambassadors), and how did it differ from previous university outreach programs (such as the Public Policy Initiative in the United States)?

Hisham: The Strategy Project, which involved more than one thousand contributors to Wikimedia projects, identified increasing participation in a number of countries as a priority – and India is the largest of these. I head up a team that's tasked with catalyzing this agenda. Given the demographic profile of India (i.e., a relatively young population) and the context of students, an education pilot was planned in India.

The India Education Program pilot grew out of the success of the Public Policy Initiative in the United States during the 2010-11 academic year. I worked closely with WMF Global Education Program Director Frank Schulenburg, Global Education Program Manager Annie Lin and US Campus Ambassador PJ Tabit on designing the pilot. One decision we took early on was to focus the pilot in one city (Pune) because we knew from the U.S. pilot that it was easier to focus on one city. (Pune has a strong heritage for education in India.)

Frank, Annie and a professor from the University of Mississippi named Bob Cummings, who all had experience with Wikipedia in the U.S. higher education system, traveled to Pune, India, to kick off the pilot. PJ stayed over for just under 3 months in Pune to help establish the pilot.

Pune's campus ambassadors in a lighter moment

Frank, Annie, and I personally went through 700 applications from people who wanted to be Campus Ambassadors in India, and we chose the best 20 candidates based on their understanding of Wikipedia (even if nascent), their ability to learn, their ability to teach, their commitment and their motivators. As in the US pilot, many had never contributed significant amounts of content to Wikipedia, but all were eager to contribute in other ways - such as teaching students how to edit. (A learning from the US pilot was that Campus Ambassadors with little prior Wikipedia-editing experience did just as well as – and sometimes better than – long-term Wikipedians when it came to performance as Campus Ambassadors, because their role is to introduce students to the basics of Wikipedia on a face-to-face level.)

I also hired Nitika Tandon to focus specifically on the India Education Program, as my role also includes other responsibilities in India.

We had a team of Online Ambassadors (most of whom were experienced Wikipedians) but a failing from the India pilot is that we were unable to bring them on-board early in the semester.

What note did foundation staff take of cultural attitudes specific to India regarding attribution, copyright, responsibility and co-operation?

Nitika and I were both born and live in India. We have an intimate understanding of India cultural attitudes. I personally don't believe that Indian culture had much bearing on this pilot. Some students in India – as elsewhere – are either lazy and plagiarize or they genuinely believe that close paraphrasing means something is no longer plagiarized. Some students in India - as newbies from elsewhere where English is not the first language – find it challenging to write in language appropriate to Wikipedia. And, some students in India – as newbies elsewhere – value responsibility and co-operation but are unfamiliar with Wikipedia culture and were unresponsive to actions & comments from others in the community taken in reaction to their edits or comments left on their talk pages.

Having said this, the real challenges were not cultural but programmatic. It's a pilot, and we've learned many important lessons. There were two that were most important, though. First, engage the community early. When the power of the community is brought to bear on a challenge, we get a huge number of solid ideas and a tremendous amount of help. It follows the open-source adage that "given enough eyes, all bugs are shallow." Second, ensure that all the infrastructure elements are in place and adequate (e.g., Campus & Online Ambassadors), selection of colleges and faculty based on involvement and capability, modifications in training of Ambassadors, faculty and students.

Do you feel you had the support, background briefing, and relationship with Wikipedian volunteers needed to address the issues that arose with the student submissions? What might have made your job(s) more effective?

I think communication with the global Wikipedian volunteer community was amongst the biggest failings of the pilot. While we had invested time and effort with the local community, we can and should have done so much better with and for the global community.

Another aspect, as I mentioned earlier, was that we had a team of Online Ambassadors to provide that crucial on-wiki role of assisting students – but, unfortunately, recruiting and training new India-focused Online Ambassadors took time, and it simply didn't happen fast enough.

We've had a lot of good interactions with the Wikipedia community through this process – we've had two IRC office hours in which we were able to explain our thinking to several editors. And we were blown away when we asked the U.S.-focused Online Ambassadors to help out with the India program cleanup too – more than 20 of them immediately offered to take on extra duties. We're so thankful that we have such a wonderful community of people who love Wikipedia as much as we do.

One point I'd like to emphasize here is that the India Education Program is a pilot program. In pilot programs, you start small (we should have started much, much smaller!), and you try different things to learn what works and what doesn't work. We adapted what had and hadn't worked from the U.S. pilot, but India is a different place, and we had a whole host of issues that the U.S. program didn't have (and we didn't have some problems the U.S. pilot had). The nature of pilot programs is to learn from them, and we've certainly learned a lot from our pilot.

When did it become apparent that there were widespread serious issues with the submissions from students in the program, and what steps or decisions did Foundation staff take in response?

We realized that at the beginning of September when a community member alerted us to the problem. Immediately, we dispatched our Campus Ambassadors to go into the classrooms and teach students about copyright violations and how to avoid doing them; Campus Ambassadors conducted more than 20 in-class sessions about copyright. Nitika and I also conducted about 15 in-class sessions with the faculty members. We pulled up students' copyright violations on the screen and showed them why this was a copyright violation and how they could fix it. Campus Ambassadors reached out to students desk-by-desk in class, by email, by text, by Facebook messages, and any other way they could think of to encourage students to stop adding copyrighted materials to Wikipedia. Our Campus Ambassadors poured their hearts and souls into telling, directing, coaxing, cajoling and begging students to not add copyvios to Wikipedia, but some students simply would not or could not understand.

By early October, we concluded that some students just weren't getting it, no matter how hard or how often we tried. We instructed all students to stop editing directly in the article namespace on Wikipedia; instead, we encouraged them to only edit in sandboxes. While we did not want students to add copyrighted materials to sandboxes either, we wanted to provide a way for us to check the students' work before it went live on Wikipedia. Many students still continued to edit in the live article namespace. Therefore, on 3 November, Nitika and I went into the College of Engineering at Pune (at Symbiosis School of Economics, deadlines had already passed for all but one class so students had stopped editing) and met with the director, who shut down the program and told students they would not be graded on anything they had added to Wikipedia after that date. The students who added good quality content would be rewarded with marks based on the quality of their edits and those who had either added plagiarized material or had not even started would lose these marks. To ensure that the problem was adequately controlled and that students actually stopped editing, the Director also stated that anyone who continues to edit will be given negative marks. (This last point is only for the remaining part of this semester.)

Did staff members feel restricted in responding to the issues by the legal/policy imperative (of the Foundation as a service provider rather than publisher) not to directly address content?

Yes. Of course we do. It breaks our hearts to see copyvios in Wikipedia text and all of us – both in India and back at the Foundation office in San Francisco – want desperately to go in and take them out ourselves, and to join in the large-scale cleanup efforts. Unfortunately, the best advice of our legal team is that we shouldn't do that, because it would be interfering in the content creation role, and could compromise our "safe harbor" immunity. These are constraints that we abide by as a result of working with / for WMF. This is another reason why we are so grateful that members of the community have volunteered to carefully check and correct the work of students.

How would you respond to the idea that it was not clear to Wikipedians who attempted to resolve the issues who was managing the project? And to their frustration as problematic submissions continued?

I'd say that from their perspective, they're probably right. We could have dealt with that using better communication processes. I'm really sorry that we didn't communicate more. This is no excuse but we spend all our time meeting with professors, Ambassadors, and students, trying to resolve issues. I wish we had communicated more with the global Wikipedia community. There were intensive efforts and communications that occurred but these were shared within the limited group of Ambassadors, faculty and students. We should have put out a lot more information on these much earlier. For instance, we could have put every important email sent by us to Ambassadors or from Ambassadors to faculty or students on public wiki pages.

I can understand the frustrations of community members. I share their frustration that some students continue to submit sub-standard material. This was one of the reasons why we decided to suspend the program at the College of Engineering Pune.

From the bottom of my heart, I thank every editor who has helped us address the quality issues coming from the India Education Program.

Do you have any personal thoughts to share on the project, or what it has revealed about the nature of university outreach and Foundation/editing community relations?

I love Wikipedia. I believe this program has considerable potential – if managed effectively – to promote participation and expand the community.

We've made mistakes in this pilot – and we've learned a great deal. While it's the nature of pilots for there to be challenges, there are some we can and should have avoided.

I also want all the issues that we have had not to cloud the wonderful contributions of so many students. We have a bunch of students who a few months ago had never edited Wikipedia, indeed didn't even know that they could edit or how to. Today, there are some remarkable contributions by them. I hope many of these students will become prolific editors going forward.

Conclusion

In light of the Foundation's significant resources, its prior experience of university outreach and the enthusiasm of applicants to become Campus Ambassadors, the failure to adequately prepare students for the task that faced them is troubling. It is not clear, for instance, why exactly the willing input of Online Ambassadors was not incorporated into the program at an early stage. One perhaps compelling explanation could lie in the hypothesis that local conceptions of copyright and attribution were so distinct from those expected on Wikipedia and so entrenched that ambassadorial evangelism could have little effect in the short term. However, Mr. Mundol’s rejection of this hypothesis makes it difficult to explain the wild discrepancy of outcomes of the IEP and the Public Policy Initiative, which only finished up in September. Yet more worrying is Mr. Mundol’s conclusion that the dramatic failings of the project are attributable to programmatic mistakes – a claim which casts doubt on Mr. Schulenberg’s position that there is no single nexus of culpability for the problems which arose and that will not fill the community’s hearts with confidence in the Foundation’s due diligence in project management. The staff members involved can be excused for not yet agreeing upon a clear determination of just what went wrong as the program winds down and focus appropriately remains on the not insignificant cleanup task, but arriving at such an analysis must surely be a strategic priority in the coming weeks.

The Wikimedia Foundation is a young and fast-growing organisation. It is pursuing with intensity laudable and ambitious strategic goals derived from an innovative and volunteer-respectful consultation process. Like its projects, it has pursued these goals with a healthy attitude towards risk, not prone to overcautiousness where gains may be made. The openness to criticism and oft-voiced declarations of appreciation for volunteer assistance of Foundation staff is commendable. This experience of the Indian Education Program, while a setback, should not result in a retreat to conservatism for outreach and expansion efforts.

Yet it is important that the Foundation give due consideration to thorough research and planning in preparation of its initiatives, and ensure these do not take for granted the patience and indulgence of the volunteer community. Failure on the former front would be to squander donor resources; and on the latter would risk fostering resentment and creating a legitimacy deficit in a climate where recent Foundation decisions – such as the non-implementation of the autoconfirmation trial, the roll-out of the Article Feedback Tool and the proposal for an image filter – have met with a difficult reception. Nor can the impact on under-prepared students and academics who were brave enough to implement a challenging and untested metric of assessment be ignored. The Indian experience offers much food for thought for future efforts. In the meantime, it may do well to bear in mind the maxim of Samuel Beckett: “Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.”

This special report was made possible by reader research and the co-operation of Foundation staff; if you have an idea for a story or opinion essay, consider informing Signpost editors on the suggestions page, at the opinion desk, or by email at wikipediasignpost@gmail.com.

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Single point of failure

  • I'd like to echo Kudpung's concern that we need to get CorenSearchBot back up and running as soon as possible. We have many policies here at Wikipedia, but only a few have legal implications and copyright adherence is one of them. Please note that the bot is fine, it's the Yahoo search engine that CorenSearchBot used that is the problem. I know the WMF staff have been trying to work with search engine providers to come up with a solution, but it's been a few months and it might be time to put more effort in to finding a solution. This also highlights another issue, single point of failure. We rely heavily on bots here at Wikipedia and certain bots, like CorenSearchBot, are critical to our operation. When those bots go down, it can be a major problem. IMHO, we should be indentifying critical bots and making sure we have a backup plan to keep them operational. - Hydroxonium (TCV) 12:53, 8 November 2011 (UTC)[reply]
Just a note: I think we're very close to having a solution to this problem. I think we'll be able to make some announcement around a week from now. Coren is involved in the resolution of this, and please know that it's being actively worked. Several staff members have put a great deal of time and energy into getting a resolution. Philippe Beaudette, Wikimedia Foundation (talk) 13:52, 8 November 2011 (UTC)[reply]
Very encouraging to hear. Skomorokh 11:09, 11 November 2011 (UTC)[reply]

Just an observation that Indian copyright law is based on UK law and the Berne convention, and is not substantially different to copyright law anywhere else on the planet. I cannot comment on whether or not there is a culture of plagiarism that is worse than in educational establishments in other places - if so, that would appear to be a structural issue that the Wikimedia Foundation cannot tackle. --Elen of the Roads (talk) 12:59, 8 November 2011 (UTC)[reply]

There's some discussion about that at the bottom of WT:IEP now, which would seem to indicate that there are some pretty serious issues with people's attitudes towards copyright in Asia (see there for full comments). The Blade of the Northern Lights (話して下さい) 15:30, 8 November 2011 (UTC)[reply]
Basically, laws and attitudes are two different things. Take laws on jaywalking for example. There are laws, but most of the people don't follow them. Same concept here. ManishEarthTalkStalk 16:40, 8 November 2011 (UTC)[reply]
The concept of plagiarism as a bad thing is quite modern. For example, ancient authors copied each other on a regular basis: what we know about the lost books of Polybius' Roman History is due to Livy's unattributed plagiarism of the earlier writer. Even as late as the 18th century, plagiarism was a regular occurrance: there is a comic incident where Benjamin Franklin, after falling out with his partner in the publishing business, then accused him of unethically printing articles from Chamber's Cyclopedia in their American newspaper -- despite the fact it was Franklin's idea in the first place! (Encyclopedia publishers in the 18th & 19th centuries plagiarized each other as a regular practice.) I believe one reason for this was that until the 19th century, a scholar considered himself very fortunate to have access to even as many books as can be found in the average high school library. Access to information & ideas is more important than giving proper credit for them; only within the last 100 years or so have we in the West achieved the luxury of abundant information, so now we expect honesty in credit. -- llywrch (talk) 17:32, 8 November 2011 (UTC)[reply]
Some modern authors (H. P. Lovecraft comes readily to mind) have encouraged people to take their ideas as well, so it's not unheard of today, but it's certainly unusual. In Lovecraft's case, it's made the copyright status of his work irredeemably confused. Totally agree with your points, though. The Blade of the Northern Lights (話して下さい) 17:52, 8 November 2011 (UTC)[reply]
Is it really useful to single out Indian student edits in this way? A much smaller, US-based initiative has had similar recent problems [1], [2]. Anyone that's worked in this area has seen copy-paste additions by editors whose user pages indicate all kinds of origins. See WP:CCI. Wouldn't it be better to just ask all course leaders to verify that they've given their students a session on plagiarism?
Another angle - some of the institutions sponsoring and/or requiring WP editing must have subscriptions to the plagiarism detector tool Turnitin. If the WP edits are done as part of an educational assignment it'd probably be acceptable from Turnitin's point of view to submit articles there for checking. Pending a re-instatement of in-house tools, which may or may not include Google books. Novickas (talk) 20:48, 8 November 2011 (UTC)[reply]
I do think it is fair to single out India (and I have worked extensively with copyright problems in Wikipedia). Though editors from all over the world post copyvios, if you look at the list of investigations at WP:CCI, a disproportionate number are from South Asia. Calliopejen1 (talk) 03:21, 10 November 2011 (UTC)[reply]
I concur with Calliopejen1. I've been working in education in Southeast Asia (with periods in India) for the last 13 years, and plagiarism is endemic here - at all levels of academia. Kudpung กุดผึ้ง (talk) 09:48, 10 November 2011 (UTC)[reply]

If you do a search with Google books, on just about any subject, you'll find books published in India in recent years (in English) that appear to have a liberal borrowing of content from various other copyrighted books. I do think twice about using a book as a reliable source if it's published in India. (Yes, I'm generalizing, but there's a lot of disregard for copyright in some places, not just India.)OttawaAC (talk) 04:45, 11 November 2011 (UTC)[reply]

Class size

From [3]:

Some of the classes enrolled for the program were very small, with only 18 students.
Smaller classes demand equal number of in-class presentations/editing sessions/refresher sessions as any bigger class would do. To get maximum impact, it makes logical sense to enroll classes with larger number of students.

Yikes. This sounds like anti-wisdom to learn from this project. I couldn't disagree more. Plagarism is a problem when mentors are not closely involved with student's work. Small class sizes and close prof / TA involvement is *vital* for getting good Wikipedia articles; lectures to the class are a dime a dozen and not that important. If the prof of an 18-person course was not familiar with Wikipedia or not monitoring their students at all, the solution is absolutely not to give them a 50-person course! If only a few profs had the knowledge / patience to do this right, then just shrink the program and keep with small class sizes. 800 students participating was part of the problem, anyway. 70 dedicated students across 3 Wikipedia-savvy professors would have done far more good than a giant haphazard program, I'm sure. If Wikipedia absolutely had to be part of a large class project... I'd still want to break it down into "labs" where a TA has a responsibility to chat with a specific set of 15-20 students, and check their work. The difference between "go to library, read refernece work on subject at hand, add passages cited to it, add new passages next week off different book from library, etc." and "sudden text dump with no referenecs" should be very obvious - IF people are paying attention early.

The other limitation on quality is that students should choose to do this at least semi-voluntarily. (I believe that the course description of Wikipedia:WikiProject Murder Madness and Mayhem mentioned the Wikipedia aspect, for example.) I'm not closely familiar with the project, but did all 800 students really know what they were getting into? Or was this a surprise homework assignment dropped on all of them? SnowFire (talk) 16:48, 8 November 2011 (UTC)[reply]

Hi SnowFire, thanks for the comment -- that page is actually an early draft of what is at meta:Wikimedia_Foundation_-_India_Programs/Education_Program, and a few things have changed, including that particular point. Our learnings will continue to evolve on the Meta page, which is more accessible than a sandbox on the English Wikipedia. :) I'll replace the content of that page with a link to Meta momentarily. I think you have a great point, though, and I'd encourage you to make comments on the talk page of the Meta page to ensure we are having community feedback on those learning points in one place. -- LiAnna Davis (WMF) (talk) 17:35, 8 November 2011 (UTC)[reply]

Get back on the horse

When you get bucked off a horse, you should get back on again as soon as you've checked out that you're still in one piece. Otherwise you're likely to start imagining difficulties and problems and how close you came to getting killed. Wikipedia is still in one piece. Similar projects should now go forward with all deliberate speed, or folks will be reluctant to get involved with this ever again.

Similar projects might include projects is other countries, say Mexico, Brazil, or South Africa. Project size should be limited, say to 100 students, until a fully successful project has been completed. I'd suggest not making the project mandatory for a grade, rather make student contributions "extra credit" assignments. Wikipedia has always been about volunteer contributors - there's no reason to change this now. Professors should review student contributions before they go into article space - that way we can know whether the problem is with the students or with the professors. Having the university administration apply for a Wikipedia grant to implement the program might help as well, by getting the top people at the university involved and putting their credibility and prestige on the line. Smallbones (talk) 17:54, 8 November 2011 (UTC)[reply]

Well said. This project tried to do too much too quickly, but experimenting with partnering with universities world-wide is exactly the kind of thing we should be doing to broaden and diversify our contributor community, and now's the time to take the lessons from the initial pilot on-board and do better. :-) --Eloquence* 19:04, 8 November 2011 (UTC)[reply]
Here's another idea; just have people edit their native language wiki instead of trying to get them to edit here. I've long advocated that we should be better, especially with Indian editors, at pointing them to their native language wikis, for many of the same reasons this project has gone awry (see the history of Malhoo for a great example of what happens when we encourage editors with very little command of English to edit en.wiki instead of their native language), and because increasing the size of the other Wikipedias will make us look more diverse and give us higher quality content everywhere, which can be translated into other languages and make better articles overall. Hindi, Tamil, Telugu, and many other Indian Wikipedias are in the low to mid 10,000s in articles, meaning they're missing a lot more than we are, and they could use the new editors more than us, not to mention the fact that the number of copyvios would probably go down because students won't feel the same pressure they do writing in a foreign language (as a Japanese student, I can relate to that pressure in some ways). I've met many Indian immigrants where I live, and most of them are great people with the best of intentions, but I would never mistake their speech or writing for Jawaharlal Nehru; there is a reason we have other language wikis, and we should make a more conscious effort to promote them. The Blade of the Northern Lights (話して下さい) 20:33, 8 November 2011 (UTC)[reply]
As far as I know, the India Edu folks have strongly promoted the existence of the Indic language Wikipedias. There's a staff person on the India team, Shiju Alex, entirely dedicated to supporting Indic language projects, and you can read a bit more about Indic language specific outreach here. In my department (engineering), we're investing significant time and effort in the development of technologies like Narayam and WebFonts, which help overcome technical barriers to participation in those languages.
As you know, the language situation in India is particularly complex. English is promoted as the lingua franca in higher ed, and it's an official language of India that's widely seen as key to professional success. At the same time, the Indic languages are also being promoted and pushed, sometimes for nationalistic reasons, or for reasons of cultural heritage. It's a very difficult context to wade into, and I think WMF is wise to generally avoid being prescriptive as to what language people should work in. My understanding -- and Nikita or Hisham would be able to add some detail on this -- is that the strong preference of the educational institutions approached in the India Edu pilot was to work in English.
I certainly do agree that we should define parameters for these programs that serve the best interests of our projects, regardless of the conditions and preferences on the ground, and decline engaging in activities that bring more harm than good.--Eloquence* 06:17, 9 November 2011 (UTC)[reply]

While understanding the reasons why the participants wanted to edit en:wp, the result has shown that many of them did not have sufficient command of English to contribute satisfactorily, particularly in technical areas where en:wp is already well-developed, and the result was not only disruptive for en:wp but must also have been extremely frustrating and dispiriting for the students. I suggest that an alternative, which would be much more likely to produce useful results, would be to translate articles from en:wp into Indic language WPs. If good-quality source articles, preferably GA or FA standard, were chosen, the receiving WPs would be improved, and the students would be able to exercise English skills in translation, would learn about article structure and sourcing and about attribution (by using the {{Translated article}} template), and would be much more likely to end with a feeling of achievement and to become long-term Wikipedians. JohnCD (talk) 12:33, 10 November 2011 (UTC)[reply]

Engaging in large scale translation projects has its very own problems. See, for example, Sodabottle's scathing criticism of Google's translation efforts in Tamil Wikipedia. Regardless of whether all the criticism is fair or not, the simple fact is that correct translation is very, very difficult, and low-quality translation is disprespectful of the Indic language communities which are trying to maintain their own standards of quality.
I don't think there are any easy answers. Growing the community of contributors in developing countries is going to be hard, no matter how we do it, and there'll be plenty of failed starts, finger pointing, and Signpost stories along the way. ;-)--Eloquence* 00:15, 12 November 2011 (UTC)[reply]
Hear, hear. Skomorokh 11:09, 11 November 2011 (UTC)[reply]
One of the reasons this project failed is a gross underestimation of the amount of work nearly 1000 novice editors (~5% of the existing body of editors) would push onto the community at large. I would be very wary of implementing these types of programs on a large scale without them being essentially self-contained–having enough supporting editors, ambassadors, and involved professors that they do not place a burden on already swamped NPPers, copyright investigators, and general cleanup crew. Danger High voltage! 22:28, 8 November 2011 (UTC)[reply]

Quoting from above:

When you get bucked off a horse, you should get back on again as soon as you've checked out that you're still in one piece. Otherwise you're likely to start imagining difficulties and problems and how close you came to getting killed. Wikipedia is still in one piece. Similar projects should now go forward with all deliberate speed, or folks will be reluctant to get involved with this ever again.

I absolutely agree with this sentiment. Take the lessons learned, try again. KConWiki (talk) 14:18, 13 November 2011 (UTC)[reply]

I am glad Smallbones suggests a smaller "pilot", though I think 100 is too large - try two or three classes, say 50. It would be a disaster to initiate another "pilot" of this size in the spirit of "get back on again". It concerns me a great deal that there has been no acknowledgment by the designers that they started out too big. If you get bucked of the horse because you didn't plan properly and overestimated your skills and capacity then simply getting back on again without acknowledging and correcting those errors is really stupid. Jojalozzo 16:08, 18 November 2011 (UTC)[reply]

This article may be of interest. Curiously, it is now at AFD. *goes to investigate* --Piotr Konieczny aka Prokonsul Piotrus| talk to me 18:07, 8 November 2011 (UTC)[reply]

An Ambassador's 2 cents

As an "Online Ambassador" for students and classes in the Public Policy project and current US education initiative, I urge the Foundation to be very careful not to bite off more than it can chew. Starting with 800 students was just 'way too ambitious. Projects need to start small in each new market to see what the issues will be before expanding. A high ratio of ambassadors to students in smaller pilot projects is essential. Perhaps this is now stating the obvious. Happy editing, everyone. -- Ssilvers (talk) 21:35, 8 November 2011 (UTC)[reply]

Even if it is stating the obvious, your point is well worth stating and re-stating. I was particularly concerned that so much responsibility, over-and-above what Campus Ambassadors are supposed to do, was placed on those bright, enthusiastic, but very inexperienced shoulders. This was compounded by bringing on a group of "special" Online Ambassadors (not chosen through the normal Online Ambassador processes) who were completely unqualified for the task. I hope too that when the IEP gets back on the horse, the organisers will reach out to the subject-specialised WikiProjects who could have provided an enormous amount of help and advice, but were never even contacted. Please don't ignore that valuable resource. Voceditenore (talk) 14:07, 9 November 2011 (UTC)[reply]

Evaluating the Pilot

Just like to say that the WMF Global Development team is taking the feedback from all sources very seriously, as we review what happened during the pilot over the past few months. We made mistakes as Frank and Hisham shared above and we are already integrating the lessons into our plans going forward. I am in the process of retaining a reporter to do a round of interviews with key actors in the process the WP community, students, professors, campus and online ambassadors and WMF staff to really capture all of the learning systematically. Her report will be shared openly with the community and I've asked her to be blunt, where necessary. While we can't go back in time, we can extract lots of learning from the pilot that will make all of our work more effective. Our team definitely plans on getting back on the horse (appreciate the sentiments Smallbones) and we plan to follow Beckett's maxim as you suggest Skomorokh. --Bnewstead (talk) 00:54, 9 November 2011 (UTC)[reply]

Glad to hear it, Barry. The Signpost will be very interested in following the review process, and wishes you the best of luck with future initiatives. Skomorokh 11:09, 11 November 2011 (UTC)[reply]

Ethics, anyone?

"Acting as a publisher of content" is a no-no. Forcing Indian kids to "contribute" is a priority. Very well. You wanted unfree labor (not exactly slavery, but not free will either), you've got unfree works (not always plagiarism, but mostly worthless). Unexpected, really?

Perhaps, if this "source" of content is really important for the Foundation, all input should be contained in an incubator or some other sort of a holding pen (and then kill them before they grow).

NVO (talk) 11:45, 10 November 2011 (UTC)[reply]

A salient point. I only edit Wikipedia with the diligence and passion that I do because I choose to do so of my own volition. Regarding the incubator: might I suggest (as I do in my point below) the Hindi Wikipedia? They could work in what will be (for most of them) their native tongue, and if there's an endemic culture of plagiarism as some other editors have been saying, then it remains confined to a wiki where those problems are dealt with as appropriate to the culture of most of that language's speakers. Brammers (talk/c) 23:27, 10 November 2011 (UTC)[reply]
Are those factors any different from the Public Policy Initiative, which produced a significant amount of worthy content? Under that analysis, any course requirements are coercive. Skomorokh 11:09, 11 November 2011 (UTC)[reply]
I do agree with Brammers that WMF should Concentrate on Indic Wikipedia and specially Hindi Wikipedia as Hindi has a very large internet user base in india as well.Nearly 40% of the population is Hindi Speaker and more than 60-70% can understand Hindi.Apart from this our main purpose should be increasing awareness regarding Wikipedia among Indians that will increase our Viewers and editors both.--Mayur (talkEmail) 11:29, 15 November 2011 (UTC)[reply]

Why en.wikipedia?

I haven't had time to rummage round the paper trail for the pilot, but why was the English language Wikipedia chosen as the target of the pilot? We have almost four million articles and stringent standards. I would have thought that more good would have come from bolstering the 100,000-ish article Hindi Wikipedia. 800 new articles there would have raised awareness of the Hindi language Wikipedia in the country where it is most relevant and increased its article count by almost 1%. In addition, there is much more low-hanging fruit to be had: whereas students creating new articles for en-wp have to scrabble around for a ridiculously niche topic, hi-wp will still have many of the more obvious potential articles still waiting to be written. Not unlike anglophones who mosey over to Simple just for the buzz of writing article after article from scratch. Brammers (talk/c) 23:19, 10 November 2011 (UTC)[reply]

Plus if the students' first language is Hindi, it would result in much better text quality than if they were to write in faltering English. And (purely conjecture here) if they can communicate an idea better in their native language, they'll probably be less likely to rip something wholesale out of a book or webpage. Brammers (talk/c) 23:22, 10 November 2011 (UTC)[reply]
So they learn the ropes at English Wikipedia and then move to their native language WP with an editing skill set. That would be a great use of the program. We're all trying to develop more serious editors here, as far as I'm concerned... The faltering first articles are immaterial. Carrite (talk)
I've often wondered about this myself - and haven't made up my mind either way. One point to keep in mind: many of the indic language editing communities are tiny and would experience the same strains as the en:wp editing community given the same circumstances. Going forward, the same lessons hold, no matter which language wikipedia is selected for the next round. Bishdatta (talk) 18:03, 13 November 2011 (UTC)[reply]
As has been mentioned elsewhere, the preferred language of instruction in higher education in India is English. While Hindi is the most common language in India, it is spoken by only about 40% of the population. While some schools may have a majority of students who speak Hindi (or some other language), it is unlikely that any institute of higher education in India will have all its students proficient in any one Indian language. One reason that English is an official language in India is that many non-Hindi speakers resist the idea that Hindi should be the common language of India. It is also the case that English is the language for access to global science, technology and commerce, which is why many Asian countries require or strongly encourage learning English, and use it as a language of instruction in higher education. -- Donald Albury 10:58, 14 November 2011 (UTC)[reply]

In defense of Indian editors

My experiences thus far with the Indian Education Project editors have been positive, mainly because out of 25 or whatever assigned projects, only a handful have made significant edits to date. So, instead of being swamped by 25 people going willy-nilly, I've been getting serious and good questions from three, and I feel like they're learning the ropes and progressing. Is their material totally smooth and clean of potential copy vio? I hope so, but y'know, we all learn as we go here. There's nothing blatant that I've seen and I feel like a few difficult economics pages have been substantially improved. And, hopefully, the editing spark has ignited a few new content creators.

The concept of getting Indian students involved is good. English Wikipedia is Anglo-American centric and getting increased content creation from other English-speaking regions of the globe is highly desirable, to my way of thinking. It's going to be a learning process for everybody. The ratio of newbie content-creators to mentors should be no greater than 5-to-1, I'll say that for sure. Expecting and forcing entire classes to contribute is a failed idea; there are probably a serious few in each class of 25 or 50 that will be willing to learn the ropes — these should be encouraged to participate. Forced participation of lower echelon students is going to result in a mess which exceeds benefit created.

The idea is good, it's a question of scale. There needs to be more one-on-one hand-holding, content creator to content creator, and less throwing shit against the wall and letting quality control supervisors pick through it for copy vio. That's not gonna work. Scale it back, keep it going. And hurray for the good Indian contributors. Carrite (talk) 04:24, 11 November 2011 (UTC)[reply]

Quality of Indian Students and Scale

While many are focusing on lack of local volunteers,Ambassadors etc, I feel these will matter very little. NASSCOM observes only 25% of students passing out are employable in Indian IT. Now if anyone has worked with Indian IT firms, the bare qualifications to secure a job in these firms is basic written / oral communication skills and Analytical abilities which is just 10th grade math. So if we make a large scale of such students participate in any program from a college we are going to get the same quality. GIGO. Srikanth (Logic) 11:45, 11 November 2011 (UTC)[reply]

Exactly. There are a top 5 or 10% of Indian students who should be targeted — or who will identify themselves by their actions — and THOSE are the ones who need to be carefully cultivated. In terms of the class project idea, I suspect the way to way to frame it is for a Wikipedia article to be a completely optional, extra credit exercise: everyone in the class registers to write on a topic, and from there the student is completely free to write or not write on that topic, according to their own wishes. Then Wikipedia mentors appear — people who actually write articles, not quality control inspectors — and offer their services. The best and the brightest, those actually interested in learning the ropes at WP will self-identify. Those who don't give a crap should not be pushed into participating. The best and the brightest who are interested then should be reassigned to other mentors, if necessary, to maintain a manageable ratio of 4-to-1 or 5-to-1, something like that. Carrite (talk) 15:43, 11 November 2011 (UTC)[reply]
By the way, this is not a dig on India or Indian students per se. I suspect that a huge percentage of students from the USA or Canada or the UK would be similarly unsuited — although probably a lower percentage, due to the fact that some Indian editors are not completely fluent in English, which makes participation at English Wikipedia problematic. Carrite (talk) 15:48, 11 November 2011 (UTC)[reply]
That's not an untested hypothesis, you know. Skomorokh 15:56, 11 November 2011 (UTC)[reply]
I agree mostly with the view expressed here. The project should target better collection of students. This may be done by targeting top institutes, such as Indian Institute of Technology or Indian Institute of Management or institutes of such stature. Although some universities included in this pilot project could be quite good, (and of course some students must have done fabulous jobs), the middle grade institutes like Symbiosis would mostly present mediocre students. Now, one or two or a few students even from such institutes may be really good contributor, but to increase the probability of getting contributors, better institutes need to be targeted. Regards.--Dwaipayan (talk) 05:26, 16 December 2011 (UTC)[reply]

Consultants' experience and exposure to Wikipedia editing, policies and guidelines

  • I don't know whether this played a part and perhaps I'm completely wrong. But did the consultants recruited have enough understanding of Wikipedia's policies and guidelines? Did they personally have established editing experience? Yes, students did commit plagiarism; but they could have been mentored at stricter levels. Just one view... Wifione Message 15:40, 31 January 2012 (UTC)[reply]

Update on the India Education Pilot

Just wanted to inform you that we have put up a post about the India Education Pilot here. Please fell free to initiate, advance or follow the conversation on the same page. Thanks

Nitika.t (talk) 10:23, 19 June 2012 (UTC)[reply]



       

The Signpost · written by many · served by Sinepost V0.9 · 🄯 CC-BY-SA 4.0