Category Archives: Assessment and Evaluation

It’s All About Your Outcomes

Posted on by
Reply

Structurally, outcomes are obligations. You need outcomes for your course syllabus, and your program as whole has some form of outcomes. From a teaching and learning perspective, however, an outcome is much more than just a hoop.  It’s at heart of why you’d bother to teach the course you do. Each outcome (and you don’t need that many), describes a skill, disposition, or set of complex knowledge that it is essential for your students to demonstrate to be successful in the course.

What does a good outcome look like?

You can read more about definitions of outcomes (what a student will do) and objectives (what an educator will teach) in another post from Gwenna Moss, but sometimes good examples can help clarify a definition.  A good outcome satisfies key criteria, including:

  • It starts with a specific, rigorous verb that reflects the type of thinking, attitude, or understanding you need students to demonstrate
  • Each outcome is worthy enough that you’ll spend a good chunk of the course returning to it and building your students’ strength with it
  • The outcome is written in language students understand and can explain in their own words

A bad outcome: Understand the definition of a just society

This outcomes is not good because there are too many ways the word “understand” could be interpreted. What would be good evidence of an outcome should be easy for students to understand the same way.  Also, this outcome might be able to be satisfied with a definition from the professor’s PowerPoint, so it isn’t worthy and long lasting enough. It is easy to make the mistake of basically describing content in your outcome, rather than what your students will demonstrate.

A much better outcome: Justify arguments about social justice using precise, accurate examples.

This is better because it specifies the type of thinking and skills student will need to do (justify an argument) and at how (using precise, accurate examples).  Social justice is a complex concept that the course will spend a long time on, deepening student conceptual knowledge overtime. Also, the skill of building an argument about social justice will built upon many multiple times in the course, sometimes in class discussions, sometime in an essay, and sometimes in an examination.  A student reads the outcome and knows the course will help you refine your skills in building arguments, and that the content will relate to social justice.

How do I write good outcomes?

  1. List the key concepts, skills, and dispositions/attitudes you’ll want in the course.  Check to ensure you aren’t just listing content.
  2. Group related things together until you have a smaller number of bigger things.
  3. Try writing statements describing things you’d accept as evidence that a student actually had the understanding, skill, or disposition you are trying to teach
  4. Look at the statements you’ve written, and ensure they each start with an active, specific verb.  Try using this list to ensure you are asking for rigorous thinking, not something students can just memorize and forget.
  5. Get someone who is not an expert to read each outcome and tell you what it says, just to make sure it is clear enough
  6. Double check that each outcome represents something you’ll want to see from students multiple times in the class. If you wouldn’t want to grade things related to it more than once, it is not important enough to be an outcome.

 

Promoting Academic Integrity: Some design questions for instructors

Posted on by
Reply

[social_share/]


Here are some propositions about students’ academic integrity that I’ve been working with:

  1. Students are more likely to do their work honestly when they see the personal value in what is to be learned.
  2. Students are more likely to do their work honestly when they believe the assessment produces actual evidence of what they have learned.
  3. Students are more likely to do their work honestly when they’ve had the chance for practice and feedback.
  4. Students are more likely to do their work honestly when they know the rules and expect them to be enforced.

Designing assessments for academic integrity is much more than tight invigilation processes and tools like Turnitin and SafeAssign (thankfully). There is much that instructors can do to set students on honest learning paths when they design and teach their courses.   Below, I offer some prompting questions instructors can ask themselves when designing course materials, assessments, and learning activities that relate to the four propositions above.

“See-the-Value” Questions for Instructors:

  • How can I convey/demonstrate the value of what students learn in my course?
  • How can I share my enthusiasm for learning this and the value I place on it?
  • How can I connect students to the benefits this learning brings for them individually, for their families or communities, for society or the world?
  • How can I provide opportunities for students to follow their individual interests and values in the context of this course?

“Evidence” Questions for Instructors:

  • What kind of evidence does this assessment provide that students have learned what I wanted them to learn?
  • What other forms of evidence could I use to determine this?
  • What alternatives could I offer students to show me what they have learned?
  • How can I make explicit to students that an assessment is transferrable to other contexts?

“Practice and Feedback” Questions for Instructors:

  • What do students need to be able self-assess their progress before grades are at stake?
  • How can I provide early feedback so that students still have the opportunity to improve?
  • How can I stage larger assignments with feedback so that students have time to improve (and avoid last minute temptations)?
  • How could I best equip students to provide feedback to each other?

“Rules” Questions for Instructors:

  • What are my rules for my assessments within the academic integrity policy framework?
  • How can I clearly explain both the assessments and the rules so that students know how to best proceed?
  • What are some common misconceptions/errors that I can address early on?
  • How can I help students learn how to follow the rules, especially when it involves technical components like a new citation or referencing protocol?
  • How can I show students that I am committed to enforce my own rules?

We have a workshop coming up at the GMCTL on November 14 that will explore assessment practices that promote academic integrity. Please consider registering.

When Performing Gets in the Way of Improving

Posted on by
Reply

[social_share/]


I encountered the following video in the spring and have been sharing it with faculty and groups with an interest in questions of assessment.  I think it lays a useful foundation for discussions on (1) what it takes to master skills and knowledge, (2) the value of lower stakes practice, (3) the necessity of formative feedback for learning, and (4) recognition that moments of “performance” or assessment for grades are also needed.

Additionally, this video supports the thinking behind a core element of the Instructional Skills Workshop—an internationally recognized workshop and certification offered regularly at our Centre.  For that workshop, participants practice the facilitation of a 10 minute “mini lesson” so valuable for improving instructional skills.  Here’s a link to more information about that workshop.

Happy to discuss the learning zone and the performance zone with Educatus readers!

First-time Thoughts on a Student Blog Assignment

Posted on by
Reply

[social_share/]


By Yin Liu, Associate Professor, Department of English
History and Future of the Book Blog

Why I did it

In 2016-2017 I taught, for the first time, a full-year (6 credit unit) English course, “History and Future of the Book,” which is one of our Foundations courses – that is, it is one of a few 200-level courses required for our majors. As in all our courses, there is a substantial writing component, usually in the form of essay assignments. I decided to complicate my life further by trying out a type of student assignment also new to me: a student-written course blog.

I had been thinking about using a student blog assignment ever since I heard a talk given by Daniel Paul O’Donnell (U Lethbridge) about using blogs in his own teaching. The point that struck me most forcibly about Dan’s argument was his observation that students wrote better when they were blogging. Since one of my goals in teaching writing is to help students write better, I thought I should give the idea a try.

Setting it up

From the outset, I had to make some fundamental decisions about how the blog assignment should work within the course. It became one of the writing assignments, taking the place of a regular (2000-word) essay: the blog post itself was to be 500-1000 words long and accompanied by a commentary (read by me only) in which students discussed the process of writing the blog post, especially the challenges they encountered and the solutions they developed. The commentary gave students a chance to reflect on and thus to learn from their own writing processes; it also helped me to evaluate the effectiveness of the assignment. The blog was made publicly available on the Web, but students could opt out of having their own work posted, although it still needed to be submitted to me for grading. Thus students also needed to supply signed permission to have their work published on the blog.

Heather Ross of the GMCTL guided me to the U of S blog service (words.usask.ca) and gave me valuable advice about permission forms and other such matters. The ICT people set me up and increased my storage quota, I fiddled with the WordPress templates, and we were good to go.

Results and learning

Each student wrote one post for the course blog, and thus the assignment was like a regular term paper except that (a) it was not an essay, and (b) it was published to the Web. Acting as the blog editor, I suggested changes to students’ first submissions, which they could incorporate into the final, published version if they wished; but I resisted the temptation to tinker with their final versions, which were published warts and all. I also used the course blog to post a series of Writing Tips for the class.

Students did, for the most part, write noticeably better in their blog posts than in their regular essay assignments. More was at stake in the blog posts; students knew that their work would be read not solely by their professor, but also by their peers and possibly by others outside the class. The informal nature of a blog also allowed students to write, in many cases, with a more genuine voice than for an essay assignment, and thus more effectively. This less formulaic, less familiar genre compelled students to rethink the basics of writing: ideas, information, audience, organisation, clarity. There was a higher chance that they would write about something that truly interested them, and quite a few expressed enthusiasm about the assignment. Students could also read and learn from the work of others in the class. The experiment was a success, and I would do it again.

Our course blog, History and Future of the Book, can be found at https://words.usask.ca/historyofthebook/. Some of the students’ posts have been removed at their request, but most remain, and you are welcome to browse through the Archive and read them – the best of them are excellent.

Open Pedagogy: Using OER to change how we teach

Posted on by
Reply

[social_share/]



There has been a considerable increase in the number of courses assigning open rather than commercial textbooks at the University of Saskatchewan.  During the 2014-2015 academic year, there were approximately 300 students enrolled in three courses using open textbooks. This year more than 2,650 students are enrolled in the at least 20 courses that have open textbooks as the assigned resource. Since the university started promoting and tracking the use of open textbooks in 2014, this use has resulted in students at the U of S saving close to $400,000 on textbook costs.

The benefits of using open textbooks and other open educational resources (OER) instead of commercial texts aren’t limited to the cost savings for students, however. The lack of copyright restrictions on OER allows instructors to modify these materials to meet the specific needs of their courses. For example, the Edwards School of Business recently released an adaption of an open book from the United States that not only saved their students money, but also meets the learning needs of the students better than the original edition. University Success will be used by the more than 475 students in the course, but also students at other institutions, and other instructors will be free to make their own changes to this resource to better meet local needs.

Just as instructors are able to adapt existing OER, so are students. Learners can become contributors to existing open materials, or use OER to create new learning materials for themselves, their peers, and future learners (and instructors).

John Kleefeld, a professor in the U of S College of Law, created an assignment that offers students in his The Art of Judgement course the chance to improve Wikipedia articles on one of the topics covered in the course. Professor Kleefeld and one of those students, Katelyn Rattray wrote an article on the design of the assignment and the experience that was published the Journal of Legal Education.

Robin DeRosa from Plymouth State University in New Hampshire created an open textbook for her early American literature course by having undergraduate research assistants find appropriate public domain content. As a core assignment in the class, students then wrote introductions for each reading based on their research about the authors and time periods.  While she served as the editor, students did much of the research and compiling of content for this new open textbook. This assignment replaced a traditional paper that would have only been seen by the instructor and the student and likely soon after marking, discarded by the student. Read more about this process on her blog.

Moving away from private “throw away” assignments can shift student activity away from knowledge consumption instead developing their skills in knowledge creation.  In the examples above and many others, this lead to increased student engagement, improved learning outcomes, and freed instructors from reading the same assignments repeatedly.

If you would like to learn more about open pedagogy, the GMCTE is offering a session on November 8 as part of our Introduction to Learning Technologies series.  You can also contact the GMCTE directly with any questions or to schedule a consultation.

Ideas about Assessing Student Participation

Posted on by
1

[social_share/]



Recently we completed another Instructional Skills Workshop (ISW) at our Centre.  This an intensive and engaging 4-day workshop where faculty and instructors learn about and practice participatory learning strategies, and upon completion, receive a certificate of completion that is nationally recognized.

As the workshop unfolds, important questions are brought forward by participants.   Given our focus on student participation in the ISW, the question of how to (and whether to) give participation marks arises.  While the answers depend on the context of the course, the teaching approach, and the design of the learning experiences and assessments, specific ideas from others can help us arrive at ways of doing this that can fit our individual teaching.

So, when I came upon a link to this blog post by David Gooblar –  I thought it was worth highlighting to our recent ISW graduates, and to readers of Educatus.

In particular Gooblar links to an article by Tony Docan-Morgan that includes a participation log template that can be found here.

We would welcome more ideas from our readers on this teaching topic.

First Day of Class: Providing students a relevant and engaging initial taste

Posted on by
Reply

[social_share/]



Sessions related to this topic will be held during the Fall Fortnight:

  • Why Teach With Top Hat? (Monday, August 22, 2016 from 10-10:25 AM) – Register Here
  • Building Student Capacity for Effective Group Work (Monday, August 22, 2016 from 1-4 PM) – Register Here 
  • Preparing & Personalizing Your Syllabus (Tuesday, August 23, 2016 from 1-2:50 PM) – Register here
  • Exploring Methods for Preventing & Detecting Plagiarism (Wednesday, August 24, 2016 from 10-11:30 AM) – Register Here
  • Attention & Memory: Increasing Student’ Learning (Friday, August 26, 2016 from 9-10 AM) – Register Here
  • Assignments, Rubrics, and Grading in Blackboard – It’s Easier Than You Think (Thursday, September 1, 2016 from 3-4:30 PM) – Register Here

As people, our perceptions and routines are engrained early on for places and people. Our experiences and decisions shaped by earlier ones. Find yourself or your students returning to their same seat? Initially excellent (or poor) meals at a restaurant tinting later dining experiences? Buying a book when the first few sentences catch your attention?

The first day of class is similar: it sets the focus of the course; instills interest; sets the context including layout and participation levels; invites selecting (or dropping) a class; and builds initial credibility and approachability.

As the recent Chronicle Vitae post by Kevin Gannon (Director of the Center for Excellence in Teaching & Learning and Professor of History at Grand View University) noted just reading the syllabus and ending class is not enough. You miss modeling how you expect students to engage in the classroom, and missing the opportunity for intriguing questions and enthusiasm for the topic. As Kevin Gannon notes “Whatever your plan for the first day, students should get some idea of what’s expected of them throughout the semester, and also have the opportunity to discern their place in the class and its activities.”

One example of engaging students in the content of the course and the level of participation and thinking you expect to see, is highlighting one of the essential ideas of the course in a way that is immediately relevant to students. Create that initial, individual, motivating connection to what they will learn.

keyholeAllow your students to glimpse the potential and be curious about where it will lead.

Highlighting a core focus of the course

1) Identify key principles, lessons or concepts that are foundational to your course

  • Resources: decoding the disciplines & threshold concept literature in nearly every discipline offers ideas to select from
  • Stats class example: ability to read descriptions of quantitative research to identify and critique reported and missing components of data description and analysis
  • Qualitative research class example: all forms of reporting involve speaking for the other person, including choosing what is conveyed and how.

2) Convey the hook of curiosity & why it is inherently meaningful to them.

  • Tip: Each discipline arose to explore and work on essential and important inquiry. As experts in the discipline, the importance is obvious but often hard to articulate to a novice.
  • Resources: Reflect on what makes this topic inherently so important and relevant. Not sure how to describe it? Try talking it out with a colleague outside your discipline (or one of us in the GMCTE).
  • Stats class example: a sample online news report of a relevant hot topic based on “research”
  • Qualitative research class example: meet and then introduce a classmate

3) Connect what students already know with what they will learn

  • Tip: Having a clear framework early on in a course allows students to organize their new ways of thinking and new information. Referring to the example, diagram, key ideas throughout the course reinforce them and help to encode new memories.
  • Resource: The meaning step in 4MAT approach to lesson planning. Attention and memory literature. There is an upcoming fortnight session on Friday August 26th.
  • Stats class example: identifying the statistical & research pieces in the news story provided and missing pieces. The initial step for critiquing.
  • Qualitative research class example: experiencing the sense of responsibility and uncertainty when speaking for another person (especially when they are sitting beside you). Wondering did they say enough or too much? Were they accurate or misinterpreted? And then connecting it with the key idea of self & research within qualitative research methods.

Photograph courtesy of David Hetre under a CC-BY license.

ePortfolios and the Curious Case of the End-of-Term Journal

Posted on by
Reply

[social_share/]


Sessions on this topic will be held during the Fall Fortnight:

  • Mahara ePortfolios (Short & Snappy session) (Monday August 22, 2016 from 11- 11:25 AM) – Register here
  • Mahara ePortfolios (Expansion Pack session)  (Tuesday August 23 from 10:30 – 11:50 PM) – Register here

As an undergrad, I took a senior studio art class in which I had to contribute something, anything, daily (well, at least weekly) to a visual journal we would hand in at the end of term. I did nothing with that journal until a stressful and long two days before it was due. My prof loved the hastily complied and craftily “aged” journal I submitted. I even pressed aged-looking coffee cup rings onto some of the pages. However, I would have gained far more from the course had I taken the time to truly focus, reflect, and learn by using the journal as a tool, than by doing nothing until the end of term.

As a masterful procrastinator, the “end-of-term journal problem” is one I think about often. How can a course in which students must produce a sustained and reflective project be structured to best enable their success?

One solution is to require regular check-ins to ensure progress is made. However, if the project is meant to be private and reflective, weekly in-person checks are drastically inefficient for an instructor, even in smaller classes.

There is another way to check progress and provide feedback: move the project online using an eportfolio. eportfolios allow students to retain their privacy while granting the instructor access to check progress and leave comments about their work. Of course, a student might still fall behind, but I would have landed 13 pages closer to completing my visual journal had I known my prof would be checking my journal regularly, prior to the deadline.

The University of Saskatchewan eportfolio tool is our own version of open sourced Mahara. It provides a tool for students to collect, reflect, and share (if they choose) their work from one or more courses. Unlike Blackboard, a student’s eportfolio remains with them between courses and they are able to customize it based on course or even program requirements. It can store and display videos, photos, documents, and text. The layout can be customized, and it has features for planning tasks and writing reflective journal entries.

In the case of my studio course, for instance, I could have used an eportfolio to post articles I was reading in my Art History classes that were informing the art I made in the studio. I could have tracked and reflected on my progress by uploading photos of each piece in different stages of completion. The possibilities are numerous, and the ability for instructors to check-in quickly on students’ work, online and from anywhere at anytime, builds in a layer of accountability and support. This layer can help instructors track students’ learning at more points in time than only due dates and exams, and help students stay engaged with ongoing projects.

3 Ideas for Promoting Academic Honesty

Posted on by
1

[social_share/]


With Elana Geller (Student Learning Services at the University Library) & Heather M. Ross (GMCTE)

A session on this topic will be held during the Fall Fortnight on Wednesday August 24, 2016. Register here.

Beauty of Reading
#1 Student skill development (Libraries)

Most students will make the right choices given enough knowledge. In order to support students attaining this knowledge the University Library maintains a number of resources including a citation guide, which can be accessed at http://libguides.usask.ca/citation. Students can also ask questions about citations at the Research Help Desk and Writing Centre, either in person or online. The Library is also looking into the creation of a tool that would have more breadth and would organize academic integrity information in an easy to use format. This tool would go beyond citation styles, to include information on collaboration, possibly specific field or discipline content, and policies. This endeavour will be one of collaboration. If you have any advice about what you would like to see in such a tool please contact Elana Geller at elana.geller@usask.ca.

#2 Technologies to detect potential plagiarism (GMCTE)

Both students and instructors have an interest in preventing and detecting potential plagiarism. For instructors, cutting and pasting questionable passages can assist in detecting materials that may have been taken from websites, journals, and other resources found online. In addition, SafeAssign is a copied text detection tool available within Blackboard. While this can be used for comparing student work to other works found online for the purpose of identifying potential plagiarism, it has great power as a teaching tool. If faculty set SafeAssign so that student can submit and then make changes based on the report, students can learn from their errors.  For more information about these issues, how to use SafeAssign following U of S guidelines, and how to use SafeAssign and Google for plagiarism detection and as learning tools, please contact Heather Ross at heather.ross@usask.ca.

#3 Assessment design (GMCTE)

When students regard what they are being asked to produce to represent relevant, valuable learning and when they believe they know what is expected and that they reasonably have the ability to do what is expected, they are more likely to invest the effort and submit authentic work for grading. With variation in disciplines for what makes an assessment appropriate and valid, not one piece of advice fits all. If you’d like to talk through some ideas for “cheat-proofing” assessments, please contact Susan Bens at susan.bens@usask.ca.

Picture courtesy of Luke Hayter and carries a CC-BY-NC license.

 

Gearing Up With Fall Fortnight 2016

Posted on by
Reply

[social_share/]



Fall Fortnight Postcard - Front“Happy New Year!!” That is how I think of September and the new school year. This often coincides with a strong pull to stationary stores, tidying my office, organizing my supplies, reading new books, and pulling out sweaters and warm socks.

Gearing up for the Fall Term is exciting. There’s often anticipation, hope, renewed energy for trying new things and looking forward to tweaking things I tried last year. I think about taking a class. There are new “school” clothes, crisp mornings, and longer shadows when I head for home. All of that is bundled together as the new term starts. I think about the new faculty, staff, and students joining the community of University of Saskatchewan in the most beautiful city in Saskatoon. And meeting new people and renewing connections with colleagues after the summer is fun.

The Fall Fortnight 2016 tugs on all these feelings of fresh starts, new ideas, learning that leads to change, connecting and reconnecting into the campus community, and gearing up for the 2016-2017 teaching and learning adventure. With over twenty sessions on a wide variety of topics in a variety of formats you will no doubt find something that intrigues you or answers a question you might have. There are Just-for-YOU sessions for new faculty, grad students, and post-docs in addition to all the other sessions on offer. New this year are sessions on the ADKAR change model and strengths-based approaches to setting up groups for success. For more highlights and a description of the sessions types take a look at this short video:

And it’s easy to register too. Check out http://www.usask.ca/gmcfortnight/

If you don’t see what you are looking for, drop us a line and let us know what you would like to see on the schedule next time around. And you can also request a tailored session—we work with you to design a session on the topic of your choice specific to your unit’s needs.

Looking forward to seeing at you at the Fall Fortnight (or in the Bowl or at a stationary store).

Fortnight Postcard - Back