An article published in the December issue of the Annals of Pharmacotherapy, "Scope, Completeness, and Accuracy of Drug Information in Wikipedia", compares Wikipedia articles on pharmaceuticals with corresponding entries in the freely accessible Medscape Drug Reference database (www.medscape.com). The report was written by four researchers: three from the Nova Southeastern University College of Pharmacy and one from the Faculty of Health and Social Work at the University of Plymouth. The report finds that Wikipedia articles on drugs (from March 12, 2008) were often incomplete in terms of side effects and drug interactions and that "Wikipedia has a more narrow scope, is less complete, and has more errors of omission versus [Medscape]". The authors conclude that consumers should not rely solely on Wikipedia for information on drugs they are taking or considering taking. However, they find that the articles had expanded considerably in scope over the 90 days before March 12 and that "user-edited sites may serve as an effective means of disseminating drug information and are promising as a means of more actively involving consumers in their own care".

The study consisted of 80 questions spread over eight "essential categories of drug information recognized as clinically important to patient safety and care": "administration, adverse drug events, contraindications, dosage, drug interactions, indications, mechanism of action, and use in pregnancy and lactation". Wikipedia articles contained answers to 32 of the questions (up from 27 questions 90 days prior), while Medscape had answers to 66 questions. However, the researchers found four incorrect or outdated answers in Medscape and none in Wikipedia; "manufacturer-provided product information and other authoritative sources" were used to generate the answer key.

In on-wiki discussions (at WikiProject Pharmacology and WikiProject Medicine), many editors have criticized the study as essentially comparing apples and oranges. MastCell summarized the central objection well:

The authors state: "The comparator database had to be a freely accessible general drug information database". Why a drug information database? They don't really explain this decision, which I think biases the rest of the study. The "sample questions" are similarly skewed – they represent clinical questions that a physician might ask, rather than common questions that a patient or layperson might seek answers to.

While acknowledging the shortcomings of many Wikipedia drug articles, editors noted the very different form, purpose and intended audience of Medscape and Wikipedia. Although accessible to the public, Medscape requires registration (free, but with no opt-out for a weekly email newsletter) to view drug monographs and some other content, and it is explicitly intended for "specialists, primary care physicians, and other health professionals". The Medscape monographs seem intended to advise health care providers on how to prescribe drugs for specific conditions; Wikipedia intentionally avoids "how-to" content and advice.

Furthermore, 10 of the missing answers in Wikipedia (which were all present in Medscape) were from questions of dosage; such information is explicitly discouraged by Wikipedia's manual of style for medicine-related articles. Similarly, the manual of style instructs editors to:

Try to avoid cloning drug formularies such as the BNF and online resources like RxList and Drugs.com. Extract the pertinent information rather than just dumping low-level facts in a big list. For example, a long list of side effects is largely useless without some idea of which are common or serious.

Part of a typical Medscape Drug Reference entry, the "Dosage, Uses and Warnings" page, consists solely of such "low-level facts", although side effects are typically sorted by frequency of occurrence.

David Ruben has been in contact with the lead author, Kevin A. Clauson; their email exchange is reproduced in the WikiProject Medicine discussion (linked above). Regarding dosage details, Clauson said:

... while I believe the Wikipedia policy to be well-intentioned, I wonder if it should be revisited. As part of my duties, I operate a Drug Information Center and over the years have received a number of calls relating to patients self medicating with prescription drugs after using online sites (not necessarily Wikipedia) as their sole information source in doing so. These patients typically are not brought to our attention until the point where something negative has happened. While I am not offering these anecdotal reports as 'evidence', there are case reports of this type of occurrence (J Am Pharm Assoc 2007 Jul–Aug;47(4):436–42), articles by researchers exploring aspects such as sharing and borrowing prescription medications (J Women's Health 2008 Sep;17(7):1073–80), and my other collaborative work with individuals whose efforts in the sociobehavioral aspects all lead me to question if Wikipedia isn't doing a disservice by electing not to provide basics such as maximum dosages? I don't know what the best answer is to that question as I have undoubtedly not given it as much thought as Wikipedia contributors have, but believing that the intended purpose of something precludes it from being used in other ways does not necessarily make it so.

Wikipedians are considering sending a letter to the editor of the Annals of Pharmacotherapy in response to the study.