Formative Feedback For Improving the Teaching and Learning Experience

Posted on by
Reply



In January of 2012 I taught my first university level course in the College of Education. It went “OK”, but not great. I had a good rapport with many of the 24 students, but no overall sense of connections with the class as a whole. Some of the comments on the SEEQ at the end of the term really surprised me and made me question myself as an educator. Teaching students is not a required part of my job, but rather an opportunity, and I was left wondering whether I wanted to let the opportunity slip away.

Instead, I looked over the SEEQ results for recurring themes and looked back on any notes that I had made throughout the term. Based on these and discussions with colleagues I made some significant changes to the course.

It’s a very hands-on course in educational technology (ETAD 470) where students learn about the technical and pedagogical aspects of using blogs, wikis, podcasts, social bookmarking and other tools for teaching and learning. I realized that a multiple choice, matching, short answer and essay question type final exam probably wasn’t a great way of assessing what I wanted the students to be learning, so I dropped it and increased how much the practical projects were worth.

learning

I also listened to the students feedback about how little value they saw in the mid-term project so I completely changed it to something they might actually have to do once teaching in a school system.

Finally, and this is really the key point of this post, I took the advice of my colleague, Kim West and implemented the use of formative feedback from the students. The class met on Tuesday and Thursdays and each Thursday I handed out a “two-minute memo” or other such slip of paper to try to gauge student understanding and concerns. On Tuesday I would go over the questions and points raised by the students on those slips (without using any names). Students had to option to leave off their names unless they had a personal note they wanted me to respond to, yet overwhelmingly students included their names, which I took as a sign of trust.

On one Thursday I forgot the slips in my office so quickly created a version using a Google Form. The students were happy with this different format so I stuck with it for the rest of the term. The responses from the students automatically ended up in a Google spreadsheet for me to easily review. I put the URL to the form on my last slide of the day and also emailed it to students (so I reached students who weren’t there that day). This would be a great way to do formative feedback in a large class as well because the collection of “data” would be automated.

I made it clear to the students at the start of the term that I had made changes based on comments from the previous year’s students and that I would be eliciting their feedback through these memos and the SEEQ for continuing changes in the class.

My impression was that the rapport in this class was far better than in the previous and this was confirmed when I received my SEEQ results. They made me feel all warm and fuzzy with comments about me as a teacher, which of course is great, but they spoke of the whole experience in the same way. They were invested in the course because A) they saw the practical assignments and activities as learning experiences that they would utilize beyond the end of the term and B) they had somewhat of a say in how the class evolved and in how it would be changed for future students.

Formative feedback like I used doesn’t have to be administered every week. It should be done as often as is appropriate to stay in touch with what your students are thinking about the learning experience (are they confused, can they hear you in the back of the room, do they love your use of examples, etc.). It should be done to find out what tweaks or larger changes you should make to your course. It should be done with an open mind. It should be done to shape the changes you think you need to make and make the teaching and learning experience better for everyone.

Weeds, Cheating and Success

Posted on by
Reply



I remember pulling weeds in a vegetable garden and coming across a strong healthy canola plant whose seed must have drifted in from the neighbour’s field. Was it a weed? If it had grown in the field then it would be considered a strong specimen, but what about in a vegetable garden? What about milkweed, wildflowers or grass?

Stray Canola
Sharing ideas and drawing on one another’s skills to reach the best answer, process check or polish a report are valued skills in the workplace and even within group projects in classes. However, during a typical test these same behaviours would be considered cheating.

In his post, UCLA Biology professor Peter Nonacs challenges us (and his students) to consider whether these behaviours are really cheating or valuable. His students faced a test where their goals for grades aligned with his goals for their learning, and the usual rule constraints were removed. The rules were no longer to sit silently and from memory write, but rather seek information you need and collaborate if you want. What do you think about what these students did and the extent they learned the game theory underlying behavioral ecology?

So what are our goals? What is a successful garden or a successful learning experience and how does our view define what is a weed and what is cheating?

By recognizing that we get to choose what are weeds and cheating when we define success, we have the power and awareness to transform our gardens and courses to what we really want them to include, including perhaps a few “weeds”.

Gather Data, Take a Timely Look, and Make Change

Posted on by
Reply



Recently, I read Fetterman, Deitz & Gesundheit’s 2010 article on using empowerment evaluation to renew a medical curriculum. The Stanford University School of Medicine engaged in a process of collecting data about their courses and providing that data back in a timely fashion to faculty and directors who engaged in reflection and discussion to create changes in courses, clerkships, and across the curriculum. Such discussions, timely feedback, and the facilitation by a curriculum evaluation person acting as critical friend were identified as key components of this process. The article provides details including how they addressed challenges for faculty such as balancing demands on time and sharing course and curriculum evaluations with colleagues to discuss strengths and weakness. The challenge of providing timely feedback within weeks rather than months was also identified and addressed (feasible with today’s computer number crunching and the personnel support for summarizing comments).

Punched cards

What struck me is the similarity between Stanford University School of Medicine’s process of empowerment evaluation and our University of Saskatchewan’s Curriculum Innovation and Renovation Cycle . Both draw on the strength of faculty members’ ability to examine data and draw on their experiences to make changes with facilitation support, and in doing so meet the principles of data-informed, faculty-driven, and developer-supported curriculum renewal process (Wolf, 2007).

The Benefits?
Improved teaching, improved student learning, improved evaluations, and a culture of evidence and reflection.

TLt 2013 Brings Ideas Together

Posted on by
Reply



The University of Saskatchewan and the Gwenna Moss Centre hosted this year’s Teaching and Learning with Technology conference on May 1 and 2. The theme was “Making IT Mainstream: Everybody’s doing IT,” focusing on “the mainstream integration of learning technologies at both the level of the institution and individual instructor; what is working and what is not, and how all of this will continue to effect higher education.”

Two pre-conference events were held on the first day; Evaluating the Integration of Technology: Understanding the Purpose and Process of Evaluation Research with Valerie Irvine, Brad Wuetherick and Stan Yu, and IDing our Future: A Meeting of the Minds of Instructional Designers, for instructional designers. Four concurrent sessions took place, with each session focusing on a four different topics and each topic featuring up to three different presenters. The diversity of presenters and topics ensured that a broad range of issues were addressed.

The conference attracted two experts in the field to speak at the opening and second plenaries. In the opening plenary, George Veletsianos shared “six research-based stories describing the integration and use of social media in higher education.”  The second plenary, titled The 21st Century University: Implications and Benefits of Access through Connections and Openness, was delivered by Valerie Irvine. The plenaries delved into interesting problems about the use of social media (Veletsianos plenary), the problem of “artificial authority” (Irvine plenary) and ubiquitous content in education, and the importance of open access to content (Irvine plenary). The conference wrapped up with a closing panel presentation.

Below are video recordings of each of the plenary talks.

George Veletsianos

Valerie Irvine

So You’re Thinking of Flipping your Class: We Can Help

Posted on by
Reply

Egg in my Roosty
Perhaps you’ve been hearing rumblings about flipped teaching. Maybe you even read my post about it in December (What is Flipped Teaching?). If you haven’t heard of it, flipped teaching is, “the process of moving lecture content from face-to-face class time to before class by assigning it as homework. This allows for more interactive and student-centred types of learning to take place during the scheduled class time. Flipped teaching often involves, but is not limited to, students viewing lecture videos as homework.”

So, now that we’re all on the same page, I am pleased to announce a new support from the Gwenna Moss Centre that we’re referring to as the “Over Easy” Flipped Experience. The first “Over Easy” experience will be offered on June 10 and 11 and will be limited to 5 participants.

“Over Easy” is intended to make the flipping process easier for instructors who would like to experiment with flipping a class. This actual experience will be partially flipped to give you the experience of being a student in a flipped course. The face-to-face portion of the experience includes two days together as a group, in which we will:

  • discuss and develop plans for flipping each unique course,
  • hear stories from instructors who have successfully flipped,
  • learn the necessary technologies,
  • learn how to find pre-created content, and
  • figure out how to make effective use of the freed up face-to-face class time.

After the two days, you will have a plan to take forward and begin flipping your class. Instructional designers from the GMCTE will be available to provide on-going one-on-one support as needed.

For more information, please visit the event page: http://www.usask.ca/gmcte/flip_overeasy. Applications are due on May 29.

Reflecting On Your SEEQ Course Evaluations

Posted on by
Reply



Most of us dread, or at the least, have mixed feelings about receiving feedback on our teaching, especially from standardized course evaluation tools such as the Student Evaluation of Educational Quality instrument (SEEQ).  However, many new insights may be gained by continuously reflecting on our actions as teachers and by opening ourselves up to a process of continual learning about pedagogy.  In a pdf booklet entitled Students Rating Teaching, teachers are challenged and encouraged to reflect upon the quality, usefulness, and potential difficulties of SEEQ data in relation to their teaching practice as opposed to simply reading their SEEQ results (Lawall n.d.).  This year when you receive your SEEQ evaluation, take a similar challenge by asking yourself, “How and why will my SEEQ data help me to continue to strengthen and improve my teaching practice?”  Here are three strategies, with accompanying questions and prompts from Lawall’s booklet, to help you begin the reflective process:

Strategy #1: Conduct a Self-Assessment 

Establish your own standards for self-assessment by completing the SEEQ instrument yourself (Lawall n.d.).  In so doing, go through the SEEQ instrument and assess your own strengths and areas for improvement, making a note of the categories that specifically connect to your teaching philosophy and goals.  Are there gaps in the instrument in terms of your teaching goals?  Should you consider adding supplementary or open-ended questions?  What is it that you want to know about what your students have learned at the end of your course?

Strategy #2: Identify Areas for Improvement and Areas of Strength

From the ratings you’ve received, in which categories did you do particularly well?  Which categories could use further refining, reflection, or improvement? At this stage, consider both types of data presented- the numeric data will provide a general view of the students’ perceptions whereas the open-ended questions often speak to individual student perceptions, beliefs, and learning styles (Lawall n.d.).  Are there any areas of divergence between your self-assessment in terms of strengths and areas of improvement versus those of your students (Lawall n.d.)?  Why or why not?

Step #3: Finding Themes

What are the major themes that appear throughout the student ratings (Lawall n.d.)?  Specifically, are there any variations or patterns you notice when comparing the numeric data with the open-ended comments (Lawall n.d.)?  Can you group the open-ended comments according to some underlying strategies or themes (Lawall n.d.)?  Which ratings speak to your teaching goals or approach and why (e.g.) motivating students through active learning, or using real-world examples in the lecture?

Once you’ve read through and reflected upon your SEEQ results, it’s important to commit to at least one or two changes that you will make to your teaching practice.  Rather than focus solely on areas of improvement, it’s important to reflect on your strengths and how you will continue to build upon these strengths in the classroom.  At this stage, it is also a good idea to connect with, brainstorm, and share strategies with colleagues as well as to re-read and/or re-write parts of your teaching philosophy statement, while incorporating any supporting evidence from your student feedback to your case file or teaching portfolio.

Reference 

You can learn more about how to reflect upon and interpret your SEEQ data from:

Lawall, M.L. n.d.  Students rating teaching: How student feedback can inform your teaching.  Retrieved February 28, 2013 from University Teaching Services, University of Manitoba at http://intranet.umanitoba.ca/academic_support/uts/media/seeq_booklet.pdf.

Mind the Gap: Learning Communities, Transitions, and Educational Enrichment

Posted on by
Reply



By Erin DeLathouwer

“What kind of job can I even get with an x degree?” I’ve heard this question again and again in my time as the program coordinator of learning communities, and I suggest that the anxiety that motivates this question comes from the fact that transitions are hard. For first-year students, for peer mentors, for new faculty, for recent graduates, and for those of us navigating from one job to another – transitions are hard.  The difficulties involved in transitions in life are mitigated, however, by education – specifically, the enriching experiences that a good education tends to provide.

In my experience, transitions aren’t supposed to be smooth; if they were, learning would be easy, the unexamined life would be valued, and growth would be stifled. Nevertheless, it can be difficult to recognize the value of education in the midst of transitions, whether those transitions are economic, personal, or educational.

One of the great challenges of the learning communities program is to inspire students to embrace the difficulties and struggles (and therefore the learning) that accompany transition to university culture. Of course some gaps are larger than others, and the gap between high school and university is too often vast. Many will not even recognize the intellectual challenge before them, for the chasm appears too wide.

To address this challenge the approach we’ve taken with learning communities has been to suggest a path which bridges some gaps so that intellectual enrichment can grow. Students in a learning community (LC) benefit from getting to know a small group of peers who take a common set of courses in their first term at university and, together with two peer mentors, explore common interests through weekly LC hours. Those common interests emerge from the unique combination of students who gather. One LC may share a common drive to get into the College of Medicine, while another may share a common interest in sustainable living; one LC might be interested in the ‘human mind’, while another might prefer exploring ‘social justice’.  Regardless of the direction students move in over the term, connections are made, thereby improving the likelihood that the intellectual challenge of university will come into view, and that students will reach for it.

Measuring the success of the learning communities program has been my greatest challenge over the years. How do we recognize and define success? If students are more likely to continue with their studies, that’s success. If students are more likely to report having had an enriching educational experience, that’s success. Both student retention and engagement have been the primary metrics by which the LC program has been assessed in its PCIP-funded[1] years. As the colleges begin to take greater responsibility for the implementation and assessment of the LC program, the challenge will be to maintain that success through this transition period for the program. The challenge will be to bear in mind the gap, bring our students’ perspectives into view, and ask ourselves how far we expect them to reach.

[1] For the past six years the Provost Committee on Integrated Planning (PCIP) has centrally funded the Learning Communities program. Over the course of the next three years that funding will be phased out with the hope that the Colleges involved will contribute to sustaining the program.

Anything But …

Posted on by
Reply



Not exams. Not this example. Not that textbook again…Anything but that!

Our rejection of a particular method or medium for teaching may be motivation enough to try something new. However, “Not ____” just rules out a single direction, leaving open all other possibilities. Deciding between the many alternatives involves setting a goal and sensing what features we want to change and what we want to retain.

For example, “Not an exam” leaves open many possibilities depending on our goals. If we want to measure students’ learning of all material in the course, we can decide to keep the end-of-term timing of final exams. A second goal of increasing writing would suggest a written report, while a goal of increasing speaking opportunities for students would point to a presentation. A goal of using an authentic assessment would have us looking for a task where students can perform relevant skills such as solving ill-defined problems that are of local or personal concern. Seeking to have an impact beyond the classroom, could lead to us creating the opportunity for students to collaborate on community-engaged projects within or beyond campus.

Scratch Golf Ball Rocket

“Not repeating last year’s material again!” also has many possibilities as to what, when, how and by whom material is reviewed. The assumption that all information needs to be repeated by the instructor during classtime through slides can be re-imagined. For example, students can create review material to share or access existing materials online, including the numeracy video created by and for Biochemistry Professor Elizabeth Shephards’ students (see the Case Study section of the University College London “Change one thing” webpage).

Stating “Anything but…” frees us to dream, but it is through visioning that we can see where we really want to go with our new freedom: even launching a rocket requires both a direction (usually up) and a magnitude of effort to create an effective force for movement.

Once we identify our goals that set the direction for our efforts, we can then choose the activities and assessments that harness the magnitude of our effort into a meaningful educational force for creating constructive alignment  within our courses and in our programs.

Looking Back, Moving Forward

Posted on by
Reply



As the academic term draws to a close and after my marking is complete, I find myself looking back over the semester, determining which learning activities went well and why, the teaching goals I set forth for myself, where I succeeded and fell short, and what I should do differently the next time I teach.  For me, the process of reflecting on my teaching practice, or “recapturing my experience, thinking about it, mulling it over, and evaluating it” (Boud, Keogh, & Walker 1985) begins on the last day of class when I ask my students to come prepared to tell me one significant thing they’ve learned or taken away from my course.  I participate in the activity too, setting the stage by explaining something new that I’ve learned that year, even though I’ve been teaching for over a decade.  I always seem to be able to find something, whether it’s seeing something I’ve taught the same old way in a new light, or learning about a new perspective, which prompts me to question how I could better present all sides of an issue.  As I listen to my students, I think about the goals that I set for myself at the beginning of the year as well as the course’s learning outcomes.  Did I accomplish what I set out to do?  Why or why not?  What will I do differently?  What resonated most with students, and why?  Did students learn what I intended them to learn?  Did they learn anything unexpected?  Why or why not?

Reflection involves looking back over the journey of where we have been, but I think most would agree it’s about moving forward too.  In the academic world, when time spent on marking final exams in April quickly turns into time on research, conferences, course planning, and other activities, it may be challenging to find the time to truly reflect.  But as Jean Koh Peters and Mark Weisberg (2011) note:

Often, when we’ve paused and picked up our heads from our grinding “to do” list, we’ve experienced something remarkable, even beautiful- the spread of the semester behind us, the long journey travelled, the deep thinking, hard work and meetings of mind that have comprised what we and our students have learned and how we’ve grown.  For teachers yearning for reflection, ending times are rich in insight and unique in opportunity; this can also be true for our students (p.172).

For teachers new to reflection, sometimes all it takes is a prompt or two, for example answering a few of the questions that I’ve posed above.  If you’re a more experienced practitioner, consider this time of year the perfect opportunity to explore the “exercises, stories, and invitations” in Koh Peters and Weisberg’s (2011), A Teacher’s Reflection Book.  As you reflect, consider ways to make reflection a learning opportunity for your students too.  How could you share what you’ve changed about your course (and why) with former students?  Think about how powerful it might be from your student’s perspectives to learn that your teacher made actual changes to their course based upon feedback received from your classmates.  By sharing the changes we have made with others, we go beyond self-assessing our own growth as teachers to modelling the powerful nature of reflection to our colleagues and students too.

References

Boud, D., Keogh, R., and Walker, D.  1985.  (eds.)  Reflection, turning experience into learning.  New York: Kogan Page.

Koh Peters, J.  and Weisberg, M. 2011.  A teacher’s reflection book: Exercises, stories, invitations.  Durham, North Carolina: Carolina Academic Press.

We Are All Treaty People

Posted on by
Reply



In the fall, the office of the VP Teaching and Learning began offering treaty education to faculty and staff at the University of Saskatchewan.  Each month, a cohort of people engaged in an online module, and then their learning culminated in a three-hour face-to-face session with a Traditional Knowledge Keeper who further illuminated treaty history and issues, and who also provided some critical cultural context.  Gordon Barnhart, in his 2007 speech to the throne, made treaty education mandatory in the K-12 school system, so our upcoming generation has some knowledge of treaty history and issues.  But for most of the rest of us, we have had little exposure to treaty education, and have a huge gap in our knowledge of First Nations peoples.

The development of the treaty module took over a year, and many people were consulted with, including Elders and people from the Office of the Treaty Commissioner.  The first question asked in every face-to-face session is, “Why do you think the university has invested in teaching faculty and staff about treaties?”  The answers are fairly consistent: it is in accordance with the goal of “Aboriginal Engagement” in the third integrated plan, many people have never learned about treaties, there is a growing demographic of First Nations people, and non-First Nations need to become better informed in order to create a more hospitable campus environment for Aboriginal students.

But for me, who has collaborated on the development and delivery of these modules, there are issues that are even more important, and these are what motivate me to learn and share what I know (so far).  I will use two Cree words to describe these important principles, from the book Treaty Elders of Saskatchewan, with the hopes that I am not using the words improperly. The first principle that has resonated for me, is that of kihci-asotamâtowin, which means “sacred promises to one another, the treaty sovereigns sacred undertakings”(25). A common misunderstanding amongst people is that the treaties were mere land transactions, undertaken and finalized over 100 years ago.  But when the new comers entered into treaties, they entered into a covenant with First Nations people, which was made sacred by the smoking of the pipe.  In doing so, it was understood that both parties were making an agreement, not only with one another, but also with the Creator.  Thus, the covenant was an enduring one, which could only be broken by the Creator (Treaty Elders of Saskatchewan, 25).  The treaties were to last as long as “the sun shines, the grass grows, and the rivers flow.”  So today, we—everyone—are responsible for honouring the treaties, not only as they were written, but also as they were orally negotiated.

The second principle that is important to me is miyo-wîcehtowin, meaning “getting along well with others, good relations, [and] expanding the circle (14).  Becoming more knowledgeable about treaty history, and about current treaty issues promises to bring about a greater understanding of First Nations (and other Aboriginal people).  In a province that, through statistics, reveals individual and systemic racism, and discrimination toward Aboriginal people, we all have a responsibility to become more informed, and this, surely, would be a beginning point for building “good relations.”

While treaty education is a means to compensate for a western education that erased or misrepresented First Nations people, it is crucial that we learn how to create a more welcoming environment for Aboriginal students. On a larger scale, we have an obligation to both take on the rights and responsibilities of the treaties, and to make steps toward repairing the relations between Aboriginal and non-Aboriginal peoples.

Cardinal, H., and Walter Hildebrandt.  (2000). Treaty Elders of Saskatchewan.  U of Calgary P: Calgary.