Taking a Fresh Approach to the Course Design Institute

[social_share/]



For more than a decade, the Gwenna Moss Centre for Teaching and Learning (GMCTL) has offered the Course Design Institute (CDI). Throughout the CDI, facilitators from the GMCTL work with instructors on developing or redeveloping a course. We go through learning about your students, writing learning outcomes, choosing teaching strategies, developing assessments, and putting it all together through constructive alignment and the blueprinting of your course.

While the CDI had been an intensive four full-day experience within one week, a few years ago we revamped it to offer it in a “flipped” mode, with participants meeting face-to-face three half days over three weeks, plus completing activities and posting to the discussion forums to provide feedback to each other in BBLearn (our learning management system). This year, we’re taking that approach and modifying it again.

On day one, Tuesday May 2, we’ll meet with participants for a half day to cover knowing your students and writing learning outcomes.

On day two, Thursday May 4, participants will choose one of three options for a day-long elective. Participants may choose from indigenization, open pedagogy, or sustainability. Lunch is included on this day.

On day three, Tuesday May 9, also a half day session, we will talk briefly about the participants’ respective experiences in their day-long elective sessions, review their learning outcomes, and talk about assessment and rubrics.

On day four, Thursday May 11, again a half day session, we will discuss constructive alignment, instructional strategies, blueprinting your course, and course syllabi.

In between the sessions, participants will need to complete activities related to what has been covered or prepare for what will be covered in the next session. Following the CDI, participants will need to complete a brief reflective paper and, once they have begun blueprinting their course, meet with one of the facilitators for a one-on-one consultation.

For more information about the CDI or to apply to participate, please see the Course Design Institute page on our website or contact me at the GMCTL.

WikiProjects, Article Importance, and Article Quality: An Intimate Relationship (2/2)

[social_share/]



In this second part of this two-part article, I discuss how WikiProjects, article importance and article quality come together in Wikipedia—and how that conjuncture can help instructors and students with selecting appropriate articles for editing in Wikipedia-based course assignments.

Understanding the Article Assessment Grid

A key WikiProject output is assessment of an article’s quality and importance (see the previous post on the criteria for measuring these). Quality and importance can be plotted on a two-dimensional grid in which each cell represents a particular quality grade and a particular importance level. The cells can then be populated with the number of articles in each of these pairings, using the quality and importance data from the article Talk pages. In Wikipedia, this information is collected—and the cells are populated—with help from a bot (more on which later), allowing for the number of articles of interest to a WikiProject, and their importance and quality, to be updated over time.

The output can be seen by looking at the assessment grid for WikiProject Adoption, fostering, orphan care and displacement (“AFOD”), discussed in the last post. I’ve captured an image of the grid as of 26 February 2017 and hyperlinked it to the actual grid on the AFOD project page, which lets you view the most up-to-date version of the grid.

For now, let’s ignore the bottom part of the grid and focus on the vertical quality axis, starting with “List.” Scanning this line, we see that there are two list-class articles, both of them considered “High” importance. Indeed, these are the only List-class articles for AFOD, as reflected in the “Total” column at far-right, which also shows “2.” Moving up to “Stub,” we can see there are three stub-class articles of high importance, 13 of mid-importance, 31 of low importance, and two that need assessing (“???”), for a total of 49 stub-class articles tracked in this WikiProject. We can keep moving up the quality axis to see the number of articles in the Start-, C-, B-, GA-, and FA-classes and their distribution across importance levels.

Now let’s look focus on the horizontal importance axis, starting with “Top.” We can see that there is one B-class article in this category, three C-class, and seven Start-class articles—for a total of 11 assessed articles in the top-importance category. Moving to the right and following the same procedure, we can see that there are 24 high-importance, 48 mid-importance, and 91 low-importance articles—with the distribution across quality levels as shown in the grid.

Assessment grid for WikiProject Adoption, fostering, orphan care and displacement (“AFOD”), as of 26 February 2017. Click on the image to see the most up-to-date version of the grid in Wikipedia.

Assessment grid for WikiProject Adoption, fostering, orphan care and displacement (“AFOD”), as of 26 February 2017. Click on the image to see the most up-to-date version of the grid in Wikipedia.

This statistical information comes together in two metrics, called WikiWork factors, that serve as a rough guide to the amount of work a given WikiProject entails. These metrics, ω and Ω (the lower- and upper-case versions of the Greek letter omega) are at the bottom of the AFOD assessment grid. The metric ω is the number of steps a WikiProject is from having all articles attain FA status; for example, an A-class article is one step away from that status, while a Stub-class article is six steps away. (List articles aren’t counted.) Multiplying the steps by the number of articles in that class and summing everything yields a ω of 986. The other metric, Ω, is a measure of relative workload: ω divided by the number of articles (again, excluding List articles).[1] It is always a number between zero and six—in this case, 4.91—with lower numbers indicating that less work is needed on average to bring an article to FA status.

Moving from the Assessment Grid to the Article Tables

The astute observer will notice that the numbers in the cells are blue, indicating a link to an active page in Wikipedia. Indeed, clicking on a number in the assessment grid to which this image links takes you to a further page that lists the article titles for the particular quality-importance pairing. (Clicking on the numbers also changes their colour, as it has done for two of the numbers in this image.) For example, clicking on the number “2” in the cell (FA, High) takes you to a table that shows the two articles that are considered “featured” (Wikipedia’s highest quality) and high-importance:

Table showing WikiProject AFOD articles rated as “High” importance and “Featured”—accessed via the (FA, High) cell in the AFOD assessment grid. Click on the image to see the most up-to-date version of the table in Wikipedia.

Table showing WikiProject AFOD articles rated as “High” importance and “Featured”—accessed via the (FA, High) cell in the AFOD assessment grid. Click on the image to see the most up-to-date version of the table in Wikipedia.

This table shows us that: (i) both articles were rated high-importance on 22 August 2009; (ii) the article “Attachment theory” became a featured article on 30 November 2009; and (iii) the article “Reactive attachment disorder” became a featured article on 18 June 2009. Clicking on the titles under the “Article” heading takes you to the current versions of the articles; clicking on the dates takes you to permanent links to the earlier versions, which, as a pink warning banner declares, “may differ significantly from the current revision.” The letters “t” and “h” are links to the Talk and History pages for the articles; the letter “l” and the heading “Score” relate to the eventual release of the article as part of the offline project, Wikipedia 1.0.

Putting it All Together: Using the Tools to Assign Articles for Editing

From the above, you can see how these tools—the assessment grid and article tables to which they lead—can be used to create a shortlist of existing Wikipedia articles for students to edit. In the AFOD assessment grid, for example, there are 62 articles rated as stub-class or start-class and considered to be top-, high- or mid-importance. This is a likely place to look for candidates for impact and improvement—though even the C- and B-class articles may also be good candidates if the aim is to achieve at least GA status. From this universe of candidates, instructors can generate a list of articles to which students can be reasonably expected to make a meaningful contribution, or from which students can choose their own articles to edit. For example, WikiProject Canadian law, of particular interest to me, has some 300 stub- or start-class articles of mid- to top-importance. The average workload, Ω, is 5.09. There is a lot of work to be done in Wikipedia—and I haven’t even begun to consider the task of writing new articles!


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former law student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621.

[1] That is, 986 ÷ (11 + 22 + 48 + 90 +30) = 4.91.

WikiProjects, Article Importance, and Article Quality: An Intimate Relationship (1/2)

[social_share/]



In a previous post, I wrote about how WikiProject Medicine acts as a forum for determining the priority (also called importance) of specific health-related Wikipedia articles and assessing their quality (also called class). More generally, these three concepts—WikiProjects, article importance, and article quality—are crucial for instructors and students to understand if they seek to use course-based assignments to improve Wikipedia. I will address each of them in turn.

WikiProjects

A WikiProject comprises a group of collaborators who aim to achieve specific Wikipedia editing goals, or to achieve goals in a specific subject or discipline represented in Wikipedia. An example of an editing type of project is WikiProject missing encyclopedic articles, which seeks to ensure that Wikipedia “has a corresponding article for every article in every other general purpose encyclopedia.” An example of a subject-specific type of project (in addition to WikiProject Medicine), is WikiProject Adoption, fostering, orphan care and displacement (“AFOD”). It aims to improve Wikipedia’s coverage of adoption, foster care, and child abandonment. A third type of project, one that combines both editing and subject goals, is exemplified by WikiProject Biography, which “concerns the creation, development, and organization of Wikipedia’s articles about persons.”

Wikipedia has a shortcut—sometimes more than one—for each project: “WP:” followed by an acronym. So, for example, entering WP:MISSING in Wikipedia’s search box takes you the Talk page for WikiProject Missing encyclopedic articles; entering WP:AFOD takes you to the Talk page for WikiProject AFOD. Entering WP:WPBIO, WP:BIOG or WP:BIOGRAPHY takes you to the WikiProject Biography Talk page. There is even a project on WikiProjects—a meta-project, if you will—with the shortcut WP:PROJ.

A WikiProject Council tracks these projects, including activity levels and inter-project discussions. According to the most recent version of the Council’s WikiProject List, There are about 2,000 WikiProjects in English Wikipedia, with varying levels of activity and interest in articles. Indeed, a Wikipedia article will often be of interest to more than one WikiProject, and a key activity of participants is to identify the WikiProjects to which the article is of interest, along with the article’s importance and class, as assigned by those projects. This is done on the article’s Talk page. For example, the article on Barnardo’s, a British charity founded to care for vulnerable children and young people, is of interest to WikiProject AFOD, which ranks it as a high-importance, C-class article. But it is also of interest to WikiProject London, a collaborative effort to improve Wikipedia’s coverage of the city of London; that project ranks it as mid-importance and start-class. (See accompanying excerpt.)

WikiProject information and rankings from Talk page for Barnardo’s (accessed 8 February 2017). Text via CC by 4.0; Anne of Green Gables image and London image originally incorporated into text via CC by 2.0.

Article Importance (Priority)

There are five levels of priority: top, high, mid, low, and NA (meaning “not an article”—that is, something other than an article, like a template or category). There is also a level for “unknown” or “needs assessing” that appears as ??? to the reader. Each WikiProject has its own criteria for these rankings. The table below shows the criteria in WikiProject Medicine, along with an example of a Wikipedia page for each ranking.

Article importance grading scheme
Label Criteria Examples
Top priority Subject is extremely important, even crucial, to medicine. Strong interest from non-professionals around the world. Usually a large subject with many associated sub-articles. Less than 1% of medicine-related articles achieve this rating. Tuberculosis or Cancer
High priority Subject is clearly notable. Subject is interesting to, or directly affects, many average readers. This category includes the most common diseases and treatments as well as major areas of specialization. Fewer than 10% of medicine-related articles achieve this rating. Coeliac disease or Mastectomy
Mid priority Normal priority for article improvement. A good article would be interesting or useful to many readers. Subject is notable within its particular specialty. This category includes most medical conditions, tests, approved drugs, medical subspecialties, well-known anatomy, and common signs and symptoms. Cholangiocarcinoma or Cramp
Low priority Article may only be included to cover a specific part of a more important article, or may be only loosely connected to medicine. Subject may be specific to one country or part of one country, such as licensing requirements or organizations. This category includes most of the following: very rare diseases, lesser-known medical signs, equipment, hospitals, individuals, historical information, publications, laws, investigational drugs, detailed genetic and physiological information, and obscure anatomical features. Leopard syndrome or Flynn effect
NA NA means Not an Article. This label is used for all pages that are not articles, such as templates, categories, and disambiguation pages. WikiProject Medicine

Source: WikiProject Medicine/Assessment (accessed 9 February 2017)

Article Quality (Class)

There are nine quality levels or classes for the typical Wikipedia article: stub, start, C-class, B-class, GA (good article), A-class, FA (featured article), List, and FL (featured list). These categories are used by the Wikipedia Version 1.0 Editorial Team for deciding how close an article is to being distribution-quality (that is, to the goal of publishing Wikipedia articles in print, CD, DVD, or a combination thereof). Some WikiProjects also use intermediate classes, such as B+. A summary of the common classes is provided below. For more detailed criteria, see the Version 1.0 assessment page; for a categorized list of articles in each class, click on the name of the class in the leftmost column.

 Stub  The article is either very short or a rough collection of information that needs much work. Stub-class articles are adequate enough to be accepted, but risk being dropped from article status altogether. The first step in improving a Stub-class article is usually the addition of referenced reasons that show why the topic is significant.
 Start  The article is developing but quite incomplete. Deficiencies may include inadequate citation to reliable sources or non-compliance with Wikipedia’s style guidelines. Raising the article to C-class typically requires further referencing, improvement in content and organization, and attention to grammar and writing style.
 C  The article is substantial, but still lacks important content or contains irrelevant material. The article should have some references to reliable sources, but may still have significant problems or require substantial cleanup. By the time an article reaches C-class, it typically has at least some infoboxes, photographs, diagrams or other media.
 B  The article is mostly complete and without major problems, but requires some further work to reach GA status. It is properly referenced to reliable sources, using inline citations. It is balanced, reasonably well written, and has a defined structure, including a lead section. Supporting materials, such as illustrations, diagrams and an infobox, should be included where relevant and useful. The article should not assume unnecessary technical background and should either avoid or explain technical terms where possible.
 GA  The article has attained good-article status (indicated at the top of the article by the “plus sign” logo) via an official review. In addition to being well written and following style guidelines, it is verifiable, contains no original research, and has no copyright violations or plagiarism. It represents viewpoints fairly, giving due weight to each, and focuses on the topic without going into unnecessary detail. It is typically illustrated with copyright-compliant images that are appropriately captioned. The article is stable in the sense of not being subject to edit wars or content disputes, though comparison with a featured article on a similar topic may show areas where content could be further developed.
 A  The article is well organized and essentially complete, having been reviewed by impartial reviewers from a WikiProject or elsewhere. GA status is not a requirement for this level, but with further tweaking or peer review, it may also be appropriate for GA or FA status.
 FA The article has attained featured article status (shown by the “star” logo at the top of the article) by passing an official review. It exemplifies the best work on Wikipedia and is distinguished by engaging and professional standards of writing, presentation and sourcing. A concise lead summarizes the topic and prepares the reader for the detail in subsequent sections that are hierarchically arranged and presented in a table of contents. Citation is extensive and consistent. The article has images and other media, where appropriate, with succinct captions and acceptable copyright status. FA-class articles may appear on Wikipedia’s home page, in a “Today’s featured article” section.
 List  The article meets the criteria of a stand-alone list, which is an article that contains primarily a list, usually consisting of links to articles in a particular subject area. List articles are often alphabetized or chronologically ordered and may also be annotated.
 FL  The article has attained featured list status. It comprehensively covers the defined scope, usually providing a complete set of items and annotations that provide useful and appropriate information about those items.

Source: Wikipedia:Version 1.0 Editorial Team/Assessment (accessed and adapted 12 February 2017)

In my next post, I will discuss how WikiProjects, article importance and article quality come together in a format that provides a convenient basis for selecting articles to edit for Wikipedia-based course assignments.


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former law student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621.

Wikipedia’s Gender Bias – and What Your Students Can Do About It

[social_share/]


Every system has its biases, and Wikipedia is no exception. A common criticism of Wikipedia is its male bias. Wikimedia Foundation, which runs Wikipedia, agreed with the criticism after it conducted a 2011 survey indicating that up to 90% of editors identified as male. This is a problem for a non-profit organization whose mission is “to empower and engage people around the world to collect and develop educational content … and to disseminate it effectively and globally.”

The mechanisms for the gender bias are various, complex, and the subject of several studies, recently summarized by two New York researchers. They may include the code-heavy interface, called wiki markup, that contributors initially had to use to edit articles. To the extent that wiki markup operated to inhibit female editors, the technical hurdle has largely disappeared: since April 2015, Wikipedia’s VisualEditor is available by default on the Article pages (but not the Talk pages) for about three-quarters of the language editions of Wikipedias. A more troubling and persistent concern may be Wikipedia’s sometimes hostile user culture, which I’ll discuss in a future blog post. In response to these concerns, there has been a series of efforts to increase female editorship. These include edit-a-thons, some organized by Wikimedia Foundation and some independently, to increase coverage of women’s topics in Wikipedia and to encourage more women to edit it. An example is the worldwide Art+Feminism edit-a-thon, the third of which was held last year to coincide with International Women’s Day. Events took place in nine locations across Canada, including Saskatoon, where editors focused on Saskatchewan and Indigenous women artists including Ruth Cuthand, Mary Longman, and Michelle LaVallee. Similar events are planned for the US, Canada and Europe in March 2017.

Anonymous Woman in Green

Woman in Green Dress, Anonymous, c 1825, National Museum in Warsaw. Source: Wikimedia Commons.

Educators and their students can help address the bias. In 2012, students in Alana Cattapan’s fourth-year seminar, “The Politics of the Canadian Women’s Movement,” edited, updated, and expanded various Wikipedia articles, including “Feminism in Canada.” Though often serving as a first point of reference on Canadian feminism, this Wikipedia page was underdeveloped, and Cattapan drew on her students to set about correcting this gateway article and other related ones. Librarians have also been active. In September 2016, to celebrate Science Literacy Week, Concordia University Library partnered with McGill Library to host a Women in Science Wikipedia event. The librarians gave a tutorial on how to edit Wikipedia, followed by an editing session in which participants got one-on-one help.

If you’re thinking of wading into the field and wondering where to start, you might want to look at WikiProject Women in Red. The goal of this project is to turn “red links”—internal links that lead to Wikipedia pages that don’t exist—into “blue links”—internal links that lead to actual Wikipedia articles. (More on this later.) Or if you’d rather start by having your students edit existing material, check out the contrapuntally-named Wikiproject Women in Green, an attempt to bring articles on women up to minimum “Good article” status. The project even provides a “Hot 99” list of women’s biographies to get you started—ranging from Aisha to Natalie Wood.


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former law student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621.

What Does It Mean to Say That “Anyone Can Edit” Wikipedia?

[social_share/]



In previous posts, I argued for the benefits of having students edit or write Wikipedia articles for university credit. But if “anyone can edit,” doesn’t that make Wikipedia prone to errors, questionable content, edit wars, and vandalism? Well, yes—these things comprise Wikipedia’s dark side and can compromise efforts at using Wikipedia for student learning. For example, Sivan Lerer, mentioned in my first post, found that when her students edited Bahá’í articles that already had substantial content, they “had a difficult time merging what they wanted with what it said in the entry,” resulting in other Wikipedians undoing their edits. But even this phenomenon can be turned into experiential learning. In “Using Wikipedia to Teach Audience, Genre and Collaboration,” Allan Bilansky, who uses Wikipedia in his social informatics course, says that “an informed effort at making contributions that persist within a large community … can be an experience at being answerable to a real audience.” Responding to a student who reported that he “had made a change [to a Wikipedia article], and then it changed back,” Bilansky told the student, “it did not change back;” rather, “[s]omeone changed it back, probably for reasons we can eventually understand.”

In both theory and practice, though, Wikipedia isn’t the freewheeling editing environment that you might think it is. Wikipedia adheres to some key principles, called the five pillars: (i) Wikipedia is an encyclopedia—not a blogging forum, a social networking site, a place to publish original research, or a dictionary (but see its companion lexical project, Wiktionary); (ii) articles adopt a neutral point of view, which includes “document[ing] and explain[ing] the major points of view, giving due weight with respect to their prominence in an impartial tone;” (iii) Wikipedia is free content that anyone can use, edit, and distribute, so anything that smacks of using Wikipedia for commercial gain attracts censure; (iv) Wikipedia has a code of conduct, or “Wikiquette,” that requires editors to treat each other with civility; and (v) Wikipedia has no firm rules, which means that “principles and spirit matter more than literal wording, and sometimes improving Wikipedia requires making exceptions.”

These principles are fleshed out in specific policies and practices. For example, a Wikipedia article should abide by three core content policies: neutral point of view (NPOV), verifiability (V), and no original research (NOR). Wikipedia is an encyclopedia, after all, which means that it is entirely derivative from other works. Thus, Wikipedians are alert to articles or edits that lack citations to reliable, published sources. Not uncommonly, articles are peppered with “citation needed” admonitions inserted by watchful Wikipedians, or prefaced by warnings that the article may violate NPOV, V, or NOR and be a candidate for deletion. Yet the civility principle means that Wikipedians must be judicious with deletions or other edits. This exhortation takes practical shape in the form of rules like the “three-revert rule”: an editor may not make more than three reversions to an article on a single page—whether involving the same or different material—within a 24-hour period.

Using view history tab on Wikipedia

Click image for screencast on using Wikipedia’s View History page.

To help students imbibe these principles and rules, Bilansky created some innovative assign­ments for his social informatics course. For example, the first assignment required students to read the View History pages (also called page history, revision history or edit history) and Talk pages (also called discussion pages) of Wikipedia articles. Every article has these pages, though most users are probably only faintly aware of them. The page history lists all the article’s previous revisions, including date and time (in UTC) of each edit, the editor’s registered username or IP address, and the user’s edit summary. As Bilansky explains, these features amount to a sort of variorum edition of each article: not only do they preserve every contributor’s minutest work and let you compare two different versions, they also provide opportunities for learning about collaborative writing processes and academic research standards.

The first thing that each of Bilansky’s students had to do was find a Wikipedia article with at least one edit that didn’t persist (that is, changes made by one editor were reverted by another), review the version history, and post to a class discussion thread an explanation of why the edits they examined were undone. (Finding such an article merely requires searching for the word reverted in the revision history.) Bilansky offered prizes for the first student to find a Wikipedia policy cited on a Talk page, which created quite a buzz and an interest in finding more strange-sounding policies or prohibitions, like the endearingly named “sock puppetry” (using multiple user accounts to hide one’s tracks). All this preparatory work smoothed the way for more complex assignments, resulting in engagement on Talk pages and in some cases, collaboration with senior Wikipedia editors (all volunteers—the whole encyclopedia is a volunteer effort) who helped the students navigate their way through what can seem like terra incognita.

I will return to the theme of Wikipedia’s integrity, because there is a lot to say about it. But in my next post, I will consider another critique of Wikipedia—its documented male bias—and what instructors and students are doing about it.


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former law student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621

 

How Students Are Learning Medicine and Collaborative Skills, And Transforming Wikipedia

[social_share/]



In my last blog post, I wrote about the wide range of disciplines represented in student Wikipedia projects. Perhaps the most ambitious effort is the Wiki Project Med Foundation, whose goal is nothing less than “to provide the sum of all medical knowledge to all people in their own language.” Started by Wikipedia enthusiast and UBC clinical professor James Heilman, the foundation is working to this goal by collaborating with various partners. These include the closely allied WikiProject Medicine, the non-profit organization Translators Without Borders, and University of California San Francisco, where fourth-year medical students have been editing Wikipedia for credit in a month-long elective course since 2013.

Amin Azzam, associate clinical professor at the UCSF School of Medicine, found that Wikipedia was second only to Google as the most frequently used source by junior physicians (!) but that there was a clear need to bring medical articles up to par. As he explains in a 2014 interview, he and his collaborators prioritized Wikipedia’s medical articles based on the number of unique visitors to the articles and the importance of the articles from a health perspective. Wikipedia also has a system for ranking article quality, ranging from “stub” to “featured article,” and the collaborators found that many articles were at the low end of the quality scale. Azzam encouraged his students to focus on the intersection of these two—high priority but low quality—and direct their efforts to improving them. Most students picked articles from this list, such as Cirrhosis and Hepatitis, while some pursued articles that held a special interest for them, like “Race and health.” Not only did they edit their chosen articles, but they reviewed articles edited by their class peers. The results, presented at a 2015 medical education conference, were impressive. As measured by Wikipedia’s own quality metrics, the students’ work resulted in improvements to most of the selected articles, and significant improvements to several.[1]

Not only has this work continued to the present, with over 50 articles improved through student work, but UCSF’s School of Pharmacy has recently joined forces with the medical school: Tina Brock, professor and associate dean of Global Health & Educational Innovations, now assigns third-year pharmacy students articles to edit from Wiki Project Pharmacology, an initiative like WikiProject Medicine. In the meantime, the Sackler School of Medicine at Tel Aviv University has also taken up the initiative. In a recent paper in Education Information Technology, educators Shani Evenstein Sigalov and Rafi Nachmias explain how their students have edited over 128 medical articles in Hebrew Wikipedia, already viewed over 1.4 million times. The paper also presents findings of a related study that focused on students’ learning experience, long-term impact and productive teaching practices.

“But surely” you might ask, “doesn’t Wikipedia’s ‘anyone can edit philosophy’ mean that all this good work can be undone?” I’ll take up that question in my next post.

[1]  Articles that went from “start class” or “C-class” to “B-class” in the first session can be considered to have improved significantly. According to the hyperlinked study, the following articles would qualify as such: “Hepatitis,” “Diabetes,” “Amyloidosis,” “Cholecystitis,” “Toxic epidermal necrolysis,” “Placental abruption,” “Therapeutic hypothermia,” “Premature rupture of membranes,” “Umbilical cord prolapse” and “Omphalitis of newborn.”


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former law student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621.

 

The Wikipedia Manifesto

[social_share/]



wikipedia-logo-v2-svg

This blog has been updated to correct some initial errors.

A spectre is haunting academia—the spectre of Wikipedia. And while there was a time when all the old powers would have entered into an alliance to exorcise this spectre, a worldwide community of educators is now taking a radically different approach: they’re assigning students the task of editing and writing Wikipedia’s sprawling content, and giving them academic credit for doing so. In the process, they’re turning students from indiscriminate knowledge consumers to savvy knowledge creators. At the same time, they’re building an open-access and up-to-date storehouse of knowledge that, in certain areas, already rivals traditional reference works. As Clay Shirky explains, this is all part of an interconnected movement—from wikis to open textbooks to interactive mapping applications like Ushahidi—in which technology has made possible methods of collaboration that never existed before.

How did this start? For some instructors, it came from a place of despair. So it was for Sivan Lerer, who teaches an introductory course on the Bahá’í Faith at Hebrew Jerusalem University. In an interview published in December 2016, Lerer explains:

I’ve told my students, for many years, that despite its many advantages (it’s accessible and in Hebrew), Wikipedia’s not an academic source . . . I told them they can use Wikipedia in the beginning but afterwards they have to go to the Encyclopedia of Islam, or The Encyclopedia of Religions. But all my admonitions were in vain. They used only Wikipedia as their source.

Lerer also wasn’t happy with the quality of writing in her students’ exams, finding that in many of their answers, they would just “regurgitate” her lectures. Sound familiar? That’s when she learned about the education program of Wikimedia Israel, whose goals are to cultivate deeper student learning and improve the access to, and quality of, Wikipedia’s resources. With the help of education coordinator Shai Katz and Darya Kantor, an active “Wikimedian,” Lerer redesigned her course to incorporate Wikipedia assignments, from revising existing articles to adding new material, such as a Canadian student’s article on the Bahá’í community in Canada. Altogether, 18 students created or improved 17 articles in Hebrew Wikipedia, with Lerer concluding that it was a positive experience for the students, and that “[i]nstead of memorizing, they really learned.”

Students have now contributed to Wikipedia as part of their course work in agriculture and life sciences, chemistry, community history, geobiology, linguistics, mineralogy, psychology, public policy, and a host of other fields. In March 2016, Wiki Edu’s Eryk Salvaggio, who writes a blog for the Wiki Education Foundation set out five reasons why you might consider assigning such a project instead of a term paper. It’s not hard to do, but before getting into the nuts and bolts of it, we’ll look at some examples of what others are doing. In particular, I want to tell you about some medicine professors who, with their students’ help, are now editing Wikipedia to make high-quality medical knowledge freely available around the world.


John Kleefeld is an associate professor at the College of Law and a 2017 teaching fellow at the Gwenna Moss Centre for Teaching Effectiveness, where he is coordinating a campus-wide project on integrating Wikipedia assignments into course materials. Portions of this blog series are from an article that he and a former student wrote about using a Wikipedia assignment for class credit. See J. Kleefeld and K. Rattray, 2016. “Write a Wikipedia Article for Law School Credit—Really?” Journal of Legal Education, 65:3, 597-621.

Hands up! How We Increase (Or Decrease) Student Participation

[social_share/]



We design courses with many opportunities for students to learn by completing assignments, readings and answering questions in class. But does our teaching increase such behaviours or decrease them?

One lens, psychology of learning, suggests we likely do both. B. F. Skinners’ operant conditioning suggests that how we respond to student behavior can either increase (reinforce) or decrease (punish) our students actions including participating in class discussion or completing homework.

What is Operant conditioning?

As Thorndike’s Law of Effect and B.F. Skinner’s operant conditioning note we are influenced by the consequences of our actions. Good consequences encourage more of this activity, while unpleasant (or unhelpful) consequences encourage less of this activity.

Reinforcement increases the frequency of behaviours through either the addition of a pleasant stimulus (positive reinforcement) or the removal of an unpleasant stimulus (Negative reinforcement).

Punishment” decreases the frequency of valued behaviours through either the addition of an unpleasant stimulus (positive punishment) or the removal of an unpleasant stimulus (negative punishment).

What about Encouraging students to answer questions in class:

Hands up! How We Increase (Or Decrease) Students Answering QuestionsWe might beneficially use punishment to decrease of disruptive behaviours such as disruptive side conversations, interrupting classmates, or answering cell phones by adding the unpleasantness of awkwardness when we stand near by, interrupt to redirect conversation, or let silence fall during the phone call.

Our effect may also be neutral leading to attenuation where the lack of a reward results in decreased responses, including when an instructor neither confirms or discounts the response and simply says “next” until they have 3 responses regardless of correctness.

Over time, behaviours do not need to be (and should not be) actively reinforced each time to maintain higher participation or lower skipping class (see information on schedules and fixed versus variable intervals and ratios).

Experiment!

Try seeing how the number of students’ answers increases (or decreases) with different responses. Predict via the lens of operant conditioning. For example:

  • What happens if I ask questions that are too easy? -> Students likely not rewarded by answering.
  • What happens if I ask questions that are too hard? ->Students might not be able to answer and receive the explicit or implicit feedback that they are wrong.
  • What happens if I present my answer(s) on a slide after I ask them? Students might not be rewarded by answering
  • But what if I skim by pointing out all the parts they identified and building on their answer? -> Students might be rewarded and increase participation.
  • What if I summarize the readings? -> Students who read now have the frustration of listening again and having “wasted time” while students who did not read are reinforced that their decision was correct.
  • What if I have them pull out the readings or use a specific page or section for an activity -> Students who read ware rewarded by not having to quickly skim, students who did not read might experience uncertainty or struggle.

Applying operant conditioning is not about “coddling” or saying “good try” without correcting flawed knowledge, but creating a learning experience that is encouraging of participation, reading and incorporating feedback into later performance. Even when a students’ answer is incorrect there are ways to reward behaviours that lead to improvement (e.g., asking questions) and provide feedback to modify that knowledge by “rewarding” the correct bits, “punishing” incorrect parts, and because we can speak better than pigeons, suggesting how to improve.

While it is useful to be cognizant of how our actions may act to encourage or discourage specific student behaviours, self-determination is still valued and people may not want themselves or others to be treated as treating people like lab rats such as by Sheldon on The Big Bang Theory:

Resources:

Tools and Strategies for “Hot Topics”- Part 3 of 3

[social_share/]



After the difficult conversation or incident in class

I regularly use formative feed-back in my class—feedback that is solicited over the course of a term, allowing me to measure student progress, highlighting concepts that are still unclear, and to hear from the students about what is and is not creating an effective learning environment. The formative feedback tool below is new to me, and will be very helpful in determining if and how the difficult topics are addressed in class were ultimately beneficial, or that they require follow-up.

The Critical Incident Questionnaire

At the end of the day (or week, or unit, or other appropriate time period), set aside 10 minutes for the class to respond, in writing, to a few specific questions.  (This may be especially helpful to do when a class session has been particularly difficult or tense).

  • At what moment were you most engaged as a learner?
  • At what moment were you most distanced as a learner?
  • What action that anyone in the room took did you find most affirming or helpful?
  • What action that anyone in the room took did you find most puzzling or confusing?
  • What surprised you most?

Keep all of the responses anonymous, and collect them at the end of the class.  Read and analyze the responses, and compile them according to similar themes and concerns.  Most important is that you report back to the group at the next meeting, and allow time for comments and discussion. (Borrowed from Vanderbilt University)

If properly prepared, you will actually find that the difficult topics and discussions allow a class to dig deeply into an issue, example, or situation that can take us beyond classroom readings and into ‘real world’ contexts, making learning even more meaningful, relevant, and transformative.

Tools and Strategies for ‘Hot Topics’ – Part 2 of 3

[social_share/]


During Class

A favorite strategy of mine has long been the “oops” and “ouch” strategy. I’m fairly certain I encountered it at a conference, but an Internet search doesn’t reveal who first developed the strategy.

In English slang, the word “ouch” means “that hurt me.” The word “oops” means, “I made a mistake” (I like to define these terms in class, in case they are not familiar to some students).

In the “The Oops and Ouch” strategy, students can express themselves in two ways. If they feel uncomfortable with something that has been said in a class or a conversation, saying “ouch” alerts their peer(s) (or instructor) that what they’ve said has hurt, offended, or angered someone, and that the original speaker needs to think about the impact their words have had. Students who have heard another person’s “ouch,” can respond with “oops,” communicating that they didn’t mean harm and/or that they’d like to think about and reword what they’ve said. During the “oops” stage, the “oops” student needs to ask genuinely curious questions to help them understand. Such a question might be, “How was what I said hurtful or offensive to you?” This opens up a constructive dialogue, and students continuously learn to see concepts and issues through a new perspective.  This strategy is one that you may want to introduce when you are co-creating your guides of conduct on your first day of class (see Part One).

Another strategy is to take the conversation away from the individuals. Sometimes it is necessary to draw the conversation away from individuals, and redirect attention toward a broader perspective, particularly if a student expresses a thought or opinion that is reflective of a more systemic myth, misconception, or belief. If a student makes a controversial statement that raises objections from other students, the original student’s statement can be more generalized by re-casting it as: “There are many people feel this way. Why do you think this is?” Then we can ask, “Many people also disagree—why do you think this is?

The Center for Research on Teaching and Learning (CRTL), University of Michigan, provides an amazing selection of resources for anticipating, facilitating, and responding to difficult discussions and moments, as well as classroom incivility:

I strongly recommend visiting each of the aforementioned links.