Practice Problem Sets: Issues of Timing and Mixing

While looking for resources for a faculty member in the sciences who was interested in incorporating more problem sets into her lectures to increase student engagement, I came upon a 2007 article by Rohere and Taylor, appearing in Instructional Science. This article describes two experiments where particular timing and mixing of mathematics practice problems improved learning.

The authors point out that it is usual for practice problems to be assigned:
• immediately following the relevant lesson (massed), and
• for problems of the same type to be grouped together (blocked).

15 - September - 2008 -- MathsThrough Rohere and Taylor’s experiments, they found that spacing the timing of two sets of practice problems 1 week apart (they called this spaced rather than massed) and varying the types of problems in a practice set (they called this mixed rather than blocked) greatly improved students’ test performance.

While the experiments used math concepts (one was a permutation task, the other was a volume task), it seems there could be an extrapolation/application to other kinds of practice problem sets for students.

The basic idea I take from this article for teaching is twofold:
(1) have students return to the problem type practiced in the previous week, and
(2) mix last week’s problem type with this week’s problem type.

This approach means students get to try a set of problems again—important especially if they had difficulty first time around. Plus, using mixtures of problems handled at the same time requires students to learn to pair each kind of problem with the appropriate procedure – that is students not only learn how to perform each procedure (learning-how), but also which procedure is for each kind of problem (learning-which) – the authors call this “discrimination training.”

A powerful closing remark in the article is that shuffling problems in this way presented few logistical demands for the teacher, making it an easy change in teaching practice that can have dramatic benefits for student learning.

(And, here I’ll add another easy change that builds on the above…teachers can ask students to do problems sets in small groups, then exchange them with another group and provide peer feedback on their calculations or choice of procedures. The learning that occurs by providing feedback to peers has also shown improvements in student performance.)

Rohere, D, & Taylor, K. (2007). The shuffling of mathematics problems improves learning. Instructional Science, 35, 381-498.

Developing ePublications

By Adrienne Thomas and Wayne Giesbrecht (Media Production)

With discussion surrounding open resources, this is a good time to talk about actually developing epublications and ebooks. For the past 3 years, Media Production (formerly eMAP) has been working with faculty and content creators to realize epub resources. With each new project, we have learned more about what to do and how to do it – an ongoing lesson as the software, media files and platforms continue to evolve.

Interprofessional Skills Learning GuideWithin the university environment, we are all concerned with the development of unique and immersive material to be used for information, education, research or knowledge mobilization purposes. If you want to make your content available as an epublication, you need to first determine who your end users are, and secondly, how they will access the material. Once you have made these decisions, it is a matter of formatting your content and designing a publication which will meet your informational or educational objectives to greatest effect. This can be a relatively easy process such as a PDF document converted to an epub format for web browser access to a more involved publication with media-rich/interactive content to be distributed across multiple supported platforms.

When thinking about access for your readers, you will also need to determine if this material will be open or free, or if it will be monetized and distributed commercially. If you self-publish, there is an opportunity for not only creative control, but price control. This was a major consideration for Dr. Bruce Grahn when he decided to e-publish his last textbook Ocular Diseases of Companion Animals for international distribution. Working with Dr. Grahn and the associated contributors, we formatted a full reflow etextbook, navigated account setups, acquired an ISBN and the required approvals (with associated proprietary file formats!) from commercial distributors. The textbook is now available for purchase on ITunes and Google Play.

The creation of the text was cost effective and any future revisions will automatically be updated in all distributed editions at no extra cost to the end user.

A media-rich experience and end user access were the requirements when we started working on an interprofessional education guide for the College of Medicine. There was a need for flexible access which would allow for independent learning or small discussion groups via mobile devices. Working with Heather Ward and Dylan Chipperfield, this project allowed us to develop an ebook which used video to moderate the content and present simulations. Embedding video within the epub presented interesting challenges, particularly for multi-platform access. When adding media-rich content (video, audio, animations, quizzes etc.), file size, reflow and platform incompatibility can be problematic and requires more consideration in layout and formatting. The project, Interprofessional Skills Learning Guide, was completed and is now accessed by health care professionals through the College of Medicine website, on ITunes and Google Play.

`Traditional publishers are invested in ebooks, it is an emerging technology likely to hold. It is also a gateway to open source educational materials and immersive experiences for students.

For more information about creating an eBook, please contact Adrienne at (306) 966-4280 or Wayne at (306) 966-4287.

How to Jump Start You Curriculum Innovation

Curriculum and teaching innovation are rarely held back, ultimately, by money alone. But, if getting a bit of money could get your initiative moving ahead, you might like some help from the Curriculum Innovation Fund.

What is the Curriculum Innovation Fund?
The CI Fund is intended to provide financial support to curriculum innovation and renewal projects at the University of Saskatchewan that specifically change or develop content or methods of a collection of courses.   The fund can support initiatives focused on a single course that pilot an innovation or show potential to contribute to program-level change.

Why was the Curriculum Innovation Fund established?
The fund was established by the Provost’s Committee on Integrated Planning (PCIP) to implement the priorities of IP3 and specifically to support innovation in academic programs and services. The fund is allocated $250K annually, beginning in 2012-2013 and ending in 2016-2017. For a list of projects and recipients to date, see:

What is the application process?
The application is in the form of a 4-page word document with word limits suggested, point form accepted, and a budget table provided. Drafting assistance is available from the staff at the GMCTE. Applications are accepted at any time and response usually occurs within 1-3 weeks.

How to start?
Contact us at the GMCTE, check out our comprehensive website, and consider coming to a workshop for hands on drafting assistance on Feburary 4 or March 4. Click here for more information and to register.

Flexibility is Key When Teaching Online

As the new year and new term kick off, I’m facing a great deal of time in front of a computer for the next few months. I’m co-teaching Introduction to Learning Technologies for the GMCTE, which includes a blended face-to-face and online component for on-campus registrants and a purely online open course for everyone else. At the same time I’m taking an online course in qualitative methods for my PhD and taking the four-week long online workshop through BCcampus on adopting open textbooks, which directly connects to both my work at the GMCTE as well as my PhD. That’s a lot of screen time, even for me.

I’m trying a balancing act around all of this, plus the rest of my work at the GMCTE, a reading course for my PhD and family life, while trying to carve out just a little bit of “me” time to keep my sanity. I’m not unique and I’m not asking for pity. I’m simply giving you an example of what an online learner may look like.

SturmanWorkshop-024-130302Whenever I speak with instructors about teaching online I stress the importance of flexibility. Frequently students taking online courses, especially at the graduate level, are working full-time jobs. They may need to do their course work in the evenings or on weekends. They may live in different time zones that may put them an hour off of your schedule, or maybe 12 hours off. They may work shift work. They may only have access to the course from a work computer or a library. English may not be their first language so it might take them longer to complete readings and post to the discussion forums, or the time for reflection that online courses can offer compared to face-to-face may make it easier for them to communicate with others. This is also true of any students who may be reticent for one reason or another to speak up in face-to-face classes.

It’s important to remember, as an instructor of an online course that you need to take care of yourself as well. If you have a large class or even a lively small class, discussion forums can become daunting. Students often have expectations that you are always connected, something that face-to-face students seem to expect more and more these days as well. Make it clear in your syllabus as to when you’re likely to reply to students emails, discussion posts and even phone messages. Some instructors will reply at almost any time, while others keep stricter online “office hours”. Pick what’s going to work for you, but keep the needs of your learners in mind.

The GMCTE offers a four-week online course on teaching online that covers these topics and provides practical solutions to many of the concerns that instructors teaching online may have. The instructional designers at the GMCTE are also well versed in teaching online and would be happy to meet with you one-on-one to discuss a course you’re teaching or are considering teaching.

Who’s in Charge? You or the Room?

ISSoTL 2014 was held this past October in Quebec City. I was attracted to the conference not just because of the theme (“Nurturing Passion and Creativity in Teaching and Learning”) but also because of the location—I had not been to Quebec City before.

Quebec City WallI walked from the Hotel Claredon, reputedly the oldest hotel in Canada, to the conference centre through the gates going from the old city to the “new” city each morning. I couldn’t help but notice how different it felt from one side of the wall to the other. The transition zone was well marked and prominent.

On the winding narrow streets of the Old City, every bend held a surprise. Roadways and sidewalks were suggestions rather than rules and the pace was different—the chance of being mowed down by speeding cars was minimal. In the new city, however, traffic was whizzing by several lanes deep and the wide streets had none of the Narrow streets of the Lower Town of Quebec Cityfriendliness of the narrow winding ones. I found that I behaved one way as a pedestrian in the old city and in another way in the new city. Thank goodness for the demarcation of the gates! They were my cue for when to change my behavior.

In the conference sessions I also found that stepping across the threshold into the space cued my expectations, and subsequent behavior, depending on the furniture arrangement and size. In the large auditorium with all 500 chairs facing “the front” I had no expectation of interacting with others. I was there to listen politely to the person at the podium. In the space with round tables and different areas of focus, I knew I would be meeting people and discussing ideas. The space arrangements cued my expectations regarding what I would be doing and what would be expected of me in that space.

Two of the sessions I attended at the conference were specifically about learning spaces. I was fascinated about how changing spaces was changing how instructors were teaching and how obvious it was that a traditional lecture space commanded and conformed “all that enter here” to be in “listening-to-expert” mode. It would, of course, take great energy and effort to counteract the norms cultivated and conveyed by the space itself in order to integrate active learning! No wonder students are shocked when they are asked to do “work” in the lecture environment—the room arrangement has cued them differently.

The two sessions I attended were about the University of Lethbridge’s LEE (Learning Environment Evaluation) project and Queen’s three specially designed active learning classrooms up and running in Ellis Hall. The 4-minute video on Ellis Hall is well-worth the time to get a quick overview.

To think about: How you interact with the spaces you inhabit? Does it shape your behavior or do you make it work for you?

We would love to know how you have taken command of the space you teach in to use strategies you know are most effective rather than letting the space dictate how you teach.

The picture of the city gates is courtesy of Smudge 9000, while the one of the narrow streets of Quebec City is courtesy of GK tramrunner229. Both pictures carry Creative Commons licenses. Details are available by clicking on the pictures.

Creativity and Innovation: An example with Soil and Art

For the past ten years, Dr. Ken Van Rees has been incorporating visual art as ateaching tool in his soil science field courses SLSC 898 and 480. Van Rees, of the Department of Soil Science, was recognized earlier this year by the Society for Teaching and Learning in Higher Education and Desire2Learn’s Innovation Award in Teaching and Learning. In the following address, delivered at this year’s Celebration of Teaching, Van Rees speaks about his innovative art and soil science classes and inspiring creativity in his students.

It’s Course Design Not Entertainment: A visit from John Boyer

On October 7, we had the pleasure at the University of Saskatchewan of welcoming John Boyer from the Geography Department at Virginia Tech to speak with us about his innovative and increasingly acclaimed approaches to teaching large classes and his approaches for motivating learning and designing assessment.  Recordings of his talks are available at these links, and are embedded at the end of this post.

1. Assessment Innovations that Reduce Cheating and Enhance Learning
2. Teaching (Really) Large Classes (Very) Well

There is some repetition between them since there were slightly different audiences in attendance at both sessions and John therefore needed to describe the format of his course each time.

I got to hang out with John all day long and it was truly inspiring and thought provoking.

A few things that stand out to me about John and his messages:

  • John was intensely and passionately clear about why he wanted students to learn about the world (he teaches an intro to World Regions).  He constantly referred to his vision and not because he was trying to persuade us of anything, but to explain why he chooses to teaches the way he does.  This guy knows his true teaching philosophy and lives by it and thrives as a teacher as a result, I’d say.
  • John has many of the same frustrations with present day student approaches and attitudes that many others report.   He’s decided, because he’s so clear on what he wants them to learn, to set high standards for their engagement and learning and then to let the students decide what to do with respect to both.  Students respond and learn as a result.
  • John faces criticism from a small number of vocal colleagues about his approach.   He says voices of detractors might have stopped him in his tracks, had they started their criticism earlier in his teaching career.  A key to his success, according to John, is that he was being innovative in teaching under the radar of his colleagues and the institution.   It wasn’t until a particular tipping point of the 2,760 person class taught in a blended format with political and arts leaders visiting or skyping in to talk with his students that he started to gain notoriety at his institution and in higher education.  By then, John knew what he was doing and had the deserved confidence and positive outcomes to face the critics.  I’m happy it worked out this way for John, but am concerned about other innovators getting “put back in their places” by group norms and power structures in academia.
  • John embodies a highly entertaining persona.  This  energy and quirkiness must surely enhance the course—but only to a point, and only for some.  I have popped on to his web site  and find his appearances and podcasts much more low key and not necessarily infused with high energy.  Make no mistake, Educatus readers, it is the design of his course that has students engaged.  He provides choice, relevance, community, and fun.  John’s personality is ultimately separate from the design and content of this course.  Let me say it again, it’s the design of this course that leads to the high levels of engagement and learning.  The design.

We’re keen at the Gwenna Moss Centre to have more discussions about some of the innovative and provocative notions brought forward by John.  Let us know what you think.

John Boyer touches Down on Tuesday at the U of S

Sometimes, the time is right to reach into the past for a “re-post”. Now is such a time to look again at the February 24, 2014 post by Susan Bens since we are in the wonderful position to be hosting John Boyer at the U of S on Tuesday, October 7.   He’ll be speaking from 2:30 – 3:30 in the GSA Commons on the very structure of assessment he uses in his huge, blended course on World Regions.

Check out this event, and other events appearing under the Academic Integrity Awareness Week Banner.


What? A Menu of Assessment Options?

By Susan Bens
I have recently come upon a few interesting ideas about the conditions we create for assessment in higher education, especially with respect to deterring academic dishonesty. Standing out to me right now is a 2013 book I’ve been reading by James Lang titled “Cheating Lessons.” This book provides inspiration, encouragement, and practical advice to teachers in higher education. Lang’s premise is that cheating is an inappropriate response by students to environments that convey an emphasis on performance within the context of extremely high stakes and where extrinsic motivators overpower the “intrinsic joy or utility of the task itself” (p. 30).Slide of a Weird Grading System

Lang points his readers to an innovative assessment practice I found quite intriguing. Professor John Boyer, in his apparently infamous World Regions class of 2,670 (!) students at Virginia Tech, affords students maximum flexibility in assessment. He structures a multi-choice assessment system that pushes students away from performance orientation and instead puts the responsibility on students to choose ways of demonstrating their learning via a point system. I highly recommend a visit to Boyer’s web page for more information on his innovative approach at

New Research Guides at the University Library: LibGuides2 Update

By Shannon Lucky, Information Technology Librarian

As we enter a new Fall semester the University Library has launched a major update to our Research Guides. These guides, built on the new LibGuides2 platform, are carefully curated selections of discipline and course specific resources combined with information on how to conduct research, writing skills, and other valuable Library tools. To explore the new guides, go to the University Library homepage and choose “Research Guides” under the Tools and Services column on the left-hand menu, or go directly to


There are 3 types of Research Guides you can find through the University Library:

  • Subject Guides are maintained by your subject librarian. These guides present carefully chosen selections of subject specific, high quality, scholarly resources, saving you and your students time by highlighting the best resources in your discipline. Browse Subject Guides.
  • Topic Guides cover how-to topics such as Finding Journal Articles, How to Evaluate Information Sources, and Citation Style Guides. They also present resources and information that are not subject specific such as Open Access, Copyright for Educators, and Research Metrics. To browse Topic Guides go to the Research Guides Homepage and choose “By Type” from the menu.
  • Course Guides are built to support a specific course or individual class. They can direct students to course specific resources including permanent links to full text articles, recommended research databases, books and other library holdings, video tutorials, and much more. They can also be used to easily create reading lists that link directly to any items that the library has in our digital or physical collections. Explore an example of a Course Guide DeDe Dawson, Science Librarian, has created for a Biology 301 class. To browse other Course Guides go to the Research Guides Homepage and choose “By Type” from the menu.

Research Guides are fully integrated with the Library collections and can also including any online resources from outside the Library that fall within copyright permissions or can be linked to from the guides. This platform is very flexible and user friendly and we encourage you to make use of this newly updated resource. If you are interested in using Research Guides as a teaching tool please get in touch with your subject librarian and let them know you want to create a Course Guide using LibGuides2.  If you have suggestions or questions about your Subject Guide please contact your subject librarian or Tell US your comments. We welcome suggestions for improving the guides and tailoring them to better serve our patrons.

Peer-to-Peer Writing Feedback: That’s what friends are for!

Peer-to-PeerPeer-to-peer writing feedback is a process by which students judge other students’ written work and produce and provide feedback to them and then, in turn, also receive feedback on their own work. By feedback, I mean commentary of a formative kind: that is, students have the chance to consider and incorporate the feedback received from their peers. Peers are not assigning grades (that would be “peer assessment”), but they may be evaluating the work using a set of provided standards or a rubric. The process can be anonymous, but it does not need to be.

This fall, I will be trying out peer-to-peer writing feedback in a course I teach on leadership and professionalism.  For me, use of this learning activity serves two related learning outcomes that I intend for students:  (1) to write more clearly and concisely, and (2) to effectively provide and receive feedback (not restricted to the area of writing or to the context of peers).

I was very encouraged this week when I came upon an article by Nicol, Thomson, and Breslin (2014) where they reported on a study of how producing peer feedback impacted learning for first year engineering students. Below, I’ve integrated and summarized some of the learning benefits that stood out to me.

Receivers of peer feedback tend to find it…

  • written in more accessible language and therefore more easily understood
  • more like dialogue than a one-way transmission and also less directive, allowing students to locate the feedback they need
  • timed to allow improvements to be made

Producers of peer feedback tend to…

  • develop a better understanding of the standards being applied, and an appreciation for the role of the summative assessor (i.e., grader)
  • compare the work of their peers to their own and benefit from the examples of other approaches to writing
  • build critical thinking capacity about both the writing of peers and their own writing

And, an overarching zinger appears in the article, where the authors quote another research team:   ‘Students seem to improve their writing more by giving comments than by receiving them’ (p. 104). To me this finding aligns with the oft-quoted saying “the best way to learn is to teach.”

Let me know via your own comments if you’d like to learn more about how I go about incorporating this learning activity into my course this term and how it turns out.  Wish me luck.

Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment& Evaluation in Higher Education, 39 (1), 102-122.

Photograph by Susan Bens