Category Archives: Uncategorized

Aligning assessment and experiential learning

Posted on by
Reply

I didn’t know what to expect as I rode the elevator up the Arts tower to interview for a research assistant position for a SOTL group. I certainly didn’t expect the wave of information and Dr. McBeth’s joyful energy. She, Harold Bull, and Sandy Bonny explained the project in a unique dialect; a mix of English and their shared academic speak. I hope they didn’t catch onto my confusion when they were throwing around the term MCQ, or multiple choice question, (which refers to the Medical Council exam in my former profession). I realized that I had quite a lot to learn if I was going to succeed in this position. I’d need to learn their language.

The scholarship of teaching and learning cluster working group shared the project through a concept map that linked “Assessment” to different kinds of students, subjects, and teaching strategies. The concept map itself was overwhelming at first but organizing information is in my skill set so creating an index was a straightforward matter. Connecting that index to resources with EndNote was quite a different affair. I closed the program angrily after multiple failed attempts to format the citations in the way I wanted. I had listened carefully and taken notes with Dr. McBeth, but the gap between theory and practice was large. With some perseverance, I am now able to bend the program to my will. It is a useful tool but like a large table saw or pliers used to pull teeth, it still frightens me.

Working with the SOTL concept map, I had identified the three areas, and their sub-topics, which the group is most interested in exploring:

  1. Examination/Assessment
    1. Ease of grading
    2. After experiential learning
    3. Multiple choice questions (MCQ)
  2. Type of Experience
    1. Designed
    2. Emergent
    3. Prompted reflection and relativistic reasoning
  3. Subject Permeability
    1. Alignment to common knowledge
    2. Access to affordances for self-teaching and tangential learning

Well I might as well move my things to the library and stay there until May. These topics cover a huge swath of pedagogical research. As I began reading, though, I soon saw that there were emerging patterns and overlaps among topics. Designed experiences overlapped with assessments. Multiple choice questions and cognition intersected. It was clear that while my index was neatly laid out in discreet cells in Microsoft Excel, the reality of the discourse was a lot more fluid and messier; more accurately reflected in the hand-written topic names, lines, and stickers of the concept map.

An interesting thing I discovered was that although I struggled at times in my methodology class in Term 1, the information and skills I learned there were useful in evaluating sources. I can ask questions and identify gaps where methodological choices aren’t outlined clearly. To be able to use my skills in a practical manner immediately after acquiring them is very exciting.

“…student views and assumptions about experiential learning and peer assessment may not align with data on actual learning.”

Currently I am focused on the topic of Examination/Assessment, which has the broadest scope of all topics identified. Two articles about student perception of experiential learning and peer assessment were intriguing to me. They make clear that student views and assumptions about experiential learning and peer assessment may not align with data on actual learning. This resonates with all the learning I’ve been doing about subjectivity/objectivity and research ethics. Our perceptions and assumptions can be very powerful but they shouldn’t be taken as dominant knowledge without further inquiry.

Some authors make strong claims about their findings even though the description of their methodological processes is lacking. Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012) assert that their findings “vindicate multiple-choice tests, at least of charges regarding their use as practice tests” (p. 1342). I am hesitant to completely agree with their claim based on this article alone because certain methodological details aren’t addressed, such as participants’ demographics and level of study. They also changed variables (feedback and timing for answering questions – those without feedback got more time to answer) in Experiment 2 and used a pool of participants from another area of the county (United States). The work of Gilbert, B. L., Banks, J., Houser, J. H. W., Rhodes, S. J., & Lees, N. D. (2014) is also lacking discussion of certain design choices such as appending their interview and questionnaire items and explicating the level of supervision and mentorship (and any variation thereof) that different students received in different settings. This doesn’t necessarily mean that the authors didn’t make very careful and thoughtful choices, but that either the description of their research needs to be amended or further study is necessary before making definitive claims.

Conversely, the work of VanShenkhof, M., Houseworth, M., McCord, M., & Lannin, J. (2018) on peer evaluation and Wilson, J. R., Yates, T. T., & Purton, K. (2018) on student perception of experiential learning assessment were both very detailed in their description of their research design and the methodological choices made.

I wonder if the variability in data presentation is reflective of the varying skills of researchers as writers. Perhaps it is more reflective of the struggle of researchers to move toward an evidence-based practice in the scholarship of teaching and learning. Maybe it is both.

While I will not be creating a nest in the library and making it my primary residence, there is still a lot to read, learn, and uncover. I look forward to journeying together with you.

Summary

· Sources must be carefully evaluated to ensure quality of research design and findings.· Delayed elaborate feedback produced a “small but significant improvement in learning in medical students” [Levant, B., Zuckert, W., & Peolo, A., (2018) p. 1002].

· Well-designed multiple-choice practice tests with detailed feedback may facilitate recall of information pertaining to incorrect alternatives, as well as correct answers [Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012)]

VanShenkhof, M., Houseworth, M., McCord, M., & Lannin, J. (2018) have created an initial perception of peer assessment (PPA) tool for researchers who are interested in studying peer assessment in experiential learning courses. They found that positive and rich peer assessment likely occurs in certain situations:

  • With heterogeneous groups
  • In a positive learning culture (created within groups and by the instructor)
  • Clear instructions and peer assessment methodology

Wilson, J. R., Yates, T. T., & Purton, K. (2018) found:

  • “Student understanding is not necessarily aligned with student engagement, depending on choice of assessment” – Journaling seemed best at demonstrating understanding but had a low engagement score by students, (p. 15).
  • Students seemed to prefer collaborative assessments – seen as having more learning value in addition to being more engaging, (p. 14).
  • This pilot indicates that student discomfort doesn’t necessarily have a negative impact on learning, (pp. 14-15)

This is the first in a series of blog posts by Lindsay Tarnowetzki. Their research assistantship is funded by and reports to the Scholarship of Teaching & Learning Aligning assessment and experiential learning cluster at USask.

Lindsay Tarnowetzki is a PhD student in the College of Education. They completed their Master’s degree at Concordia University in Communication (Media) Studies and Undergraduate degree in English at the University of Saskatchewan. They worked at the Clinical Learning Resource Centre at the University of Saskatchewan for three years as a Simulated Patient Educator. They are interested in narrative and as it relates to social power structures. Lindsay shares a home with their brother and one spoiled cat named Peachy Keen.

 

How do I internationalize my course?

Posted on by
Reply

Self-reflection

Step 1: Know my position and privilege. Who am I as a teacher? (This idea isn’t new, check out this article from 1958: Teacher, Know Thyself)

Step 2: Does the way I design my course plan for access and diversity?

Step 3: Do I want to “add-on”, “infuse”, or “transform” my course through internationalization?

Some direction

If you are working on step 3, there is an excellent resource of teaching tips here: Strategies for Course Internationalization. Centre for Teaching Excellence, University of Waterloo.

A simple way to start internationalization is to add assigned readings from international perspectives. This can be a way to start conversations and look for similarities and differences in findings. Even the writing and presentation structure might reflect cultural differences.

Next, take a look at your course outcomes – are students expected to develop or use intercultural competencies? How might the next version of your course highlight internationalized or global community skills?

Onwards on this journey, it’s time to look at evaluation. Inclusive assessment should include students using a metacognitive process to track their development. If that sentence doesn’t make sense on first reading, try this: a student needs to be able to know what they know and how they know it at any stage of learning. If they are just beginning, they should be able to identify that, recognize when they are building knowledge/skills/attitudes, and ultimately know when they’ve mastered or achieved the outcome of the learning. When students are involved in the assessment process, they are demonstrating choice, responsibility, and reflection. These are all attributes of inclusive learning which is fundamental to internationalization.

Here is another list of tips and tricks to start internationalizing your course.

This post is part of a series in internationalization. You can follow along here.

Come say hi! We’re at the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops to meet your needs.

 

How Might Intercultural Capacity inform our Teaching?

Posted on by
Reply

Once we develop the capacity for intercultural competence, we can start to infuse the associated skills into our teaching practice. This can take many forms but all the elements connect to the group of knowledge and skills we associate with facilitation.  Pedogogy is the study of leading learners and facilitators make a process easier. So, facilitation is the process we use to make the learning possible. In adult education, we know that our learners come with valuable prior knowledge, skill, and experience. We can draw on these to enhance the learning experience for both instructor-facilitator and student.

How do we start facilitating?

As an instructor, or facilitator, you may wish to try some of these summarized strategies suggested by Dimitrov & Haque (2017) :

When can we use it?

Using your intercultural capacity can happen any time you are interacting with someone else. Everyone has inherent uniqueness that together creates diversity.  As our ‘village’ grows, our diversity along with it.

“…more and more of us do not live in closed circles of like-minded, similarly raised people. Think of the last few gathering you attended – a work meeting, a class, a trade show. Chances are, you sat next to and talked with people from places other than where you’re from, people with different cultural norms, people of different races and religions and histories. And chances are, therefore, that you sat next to people who do practice etiquette – but etiquette different from yours, and perhaps in conflict with it on certain points.”

 – Parker, Priya (2018). The art of gathering: How we meet and why it matters. New York : Riverhead Books.

Why do we care?

Figure 1 Depiction of instructor trying to introduce too many new strategies at once. Illustration by author.

We care about using good facilitation because we want our learners to achieve the desired outcomes of the course in the most efficient and effective way possible. This means using strategies we know will allow learners to thrive. As instructors, we want to leverage the learner’s pre-existing knowledge, skills, and attitudes to make the new learning accessible and within reach. If the learning curve is too steep, learners may just fall off. And even for the instructor/facilitator, try to keep a growth mindset that we are all working to get along and want to be successful in our relationships. A positive disposition and honesty about one’s own positionality and areas for growth will go a long way!

If you’re looking for more help with developing your intercultural capacity, please reach out to the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops or short courses that can meet your needs.

NB: If you tried the text verification link above, hemingwayapp.com, you may be interested to know that this article is at a grade 10 reading level and 9 of the 39 sentences are “very hard to read”. Ideally, public text should be at a grade 9 level 🙂

All Aligned – Outcomes

Posted on by
Reply

This post is one of a 3 part series on the concept of alignment of what you want students to learn, how you plan to teach them, and what you will assess them on.  Sometimes called constructive alignment, it has three parts:

  • Your learning outcomes
  • Your instructional approach or learning strategies
  • Your assessments

This post focuses on the need for clear learning outcomes for your students, and the next two posts in October and November focus on instruction and assessment respectively.

Why outcomes

Outcomes are statements that describe what our teaching is designed to help students know, do, or be. They start with a verb, then connect that to the key content.  Knowing what they are helps us design instruction that is focused exactly on what we want.

For example, if your goal is to have students

  • Articulate a well-defended argument based on precedent

Then you want to teach them to

  • Use criteria for well-defended to assess and build arguments
  • Find and make sense of relevant precedent

Without a clear outcome about being able to actually make a well-defended argument, you might think your instruction should stop at describing criteria for well-defended, or that a test where students recalled the definition might be better than an assessment where they actually had to make an argument.

Good outcomes help us focus our classes on the knowledge, skills, and values we actually care about our students learning, rather than explaining facts and ideas.  The educator commitments in Our Learning Charter describe being “aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity” as a key shared commitment of those of us who teach on campus.

More about outcomes

Read the other chats related to Our Learning Charter to learn about other educator commitments.

 

Building Broad Minds: Active learning strategies for large classrooms

Posted on by
1

Building broad minds is not about back filling.  Broad minds are the byproduct of encountering diverse ideas, thinking deeply about them, and integrating those ideas into our own worldviews and cognitive frameworks.  In higher education, the opportunity to be exposed to the thinking of a wide variety of disciplines usually happens at the first year level. However, those are also often large courses where the primary method of instruction is listening to your professor speak.  To actually get broad minds, our learning activities have to be active, even in the large classrooms where active learning strategies are limited by the room, and even when students are first encountering the subject mater.

A great simple rule for broad minds is the 10:2 ratio. It basically means that for every 10 minutes of lecture, and student needs 2 minute of social processing to make sense of it. Lots of the time, we think group work in classes is all about assignments.  Actually, it is much more about helping us make sense of what we learning. To encourage broad thinking, consider pausing at least every 10 minutes and doing a short activity that allows student to make sense of what you’ve just tried to teach them.

Use daily active collaboration in increase understanding

  • Having students problem solve in pairs
  • Having students turn and talk to each other about the implications of a new idea you introduced or why it matters
  • Provide opportunities to try something and get feedback from a peer. You don’t have time to provide all that feedback in a large class, but feedback is helpful in both improving and remembering, and there are many other people in the room who can help.  Giving feedback also refines our understanding, so your students are learning when giving and receiving feedback.
  • Have students teach each other something quick (not a big group presentation). Read more from the research about why this is one of the best ways to improve student understanding (Topping and Stewart, 1998).

Have students engage in critical thinking and think from multiple worldviews
Developing broad minds requires encountering the major debates of the discipline early and considering them from multiple perspective.  Although it is hard to have teams of two students debate other teams of two students in a large lecture theater, all students can think and speak simultaneously.  Want to see how it would work?

 

Learn more:

Graduates with perspectives and approaches the world needs

Posted on by
2

We often talk about the skills our graduates will need for success in their work and within our communities. As we aspire to be the university the world needs, we can’t overlook how essential perspective taking and cross-cultural competence are in our increasingly diverse world.  In this place, we have a collective commitment to improve the situation for the First Nation, Metis, and Inuit peoples, and to truth and reconciliation. And we can also see the impacts of nationalism and nativism on the global stage, A problem that is prompting us to equip our students with the skills they will need to respond.This post is one in a series related to the educator commitments in Our Learning Charter. It focuses on how to help students to explicitly recognize their own position and work to understand, acknowledge, and value perspectives and worldviews different from their own.

What you can model in your teaching:

  1. Start by acknowledging your own position and privilege. Being a role model and ally is essential in supporting students in the process of doing the same thing. Knowing why you would include a land acknowledgement, for example, rather than omitting one or just adding one to your syllabus is part of an acknowledgement. Not quite sure how to approach it in a good way? Join the short course in the fall on Indigenization, decolonization and reconciliation at the Gwenna Moss Center for Teaching and Learning (GMCTL).
  2. Purposefully include content, perspectives, and worldview from local Indigenous communities and international perspectives.  The focus should be on being prepared to support a diverse world and set of different views. Need some support?  Ask for a consultation at GMCTL.
  3. Deliberately offer more than one perspective on the debates of your discipline whenever possible, and explain the value of those discussions to the disciplinary discourse. Provide opportunities for students to engage in facilitated discussions about those debates without taking a position yourself.

What you could do with your students:

  1. Choose to share your own power by using active learning strategies to get students thinking and talking, rather than transmission styles where students mostly listen. Understanding,  acknowledging, and valuing perspectives and worldviews different from their own is requires active learning processes, because it requires students to be in dialogue with the other.  Learn more about the research on active learning or experiment with some active learning strategies in your class.
  2. Provide students with deliberate opportunities to work in culturally diverse groups where they’ll be exposed to a multiplicity of perspectives that they might not encounter, given that we are more likely to self-select groups of people like us.
  3. Proactively plan for how to have challenging conversations with students in class, and how to respond when students struggle to value worldviews and perspectives other than their own.  Not sure how do this? Join the GMCTL for a workshop series on planning for and responding to difficult conversations in the classroom or preview some online resources.

View other posts in the Charter Chat series.

Charter Chats

Posted on by
2

The University of Saskatchewan has a new Learning Charter.  First written in 2010, then updated in 2018, our charter is helpful in framing what we believe about teaching and learning.  It will also provide a sense of direction for work at Gwenna Moss over the next year:

The Learning Charter thus acts as a conceptual map and planning document, linking together our pursuits and how we strive for them, encouraging and guiding us on our educational journey. As a map, it is also a focal point for our community to discuss where we are and where we want to go in our shared future.

This post will be the hub of the “Charter Chats” related to our new charter.  As each new monthly post is written, the post will be linked on this page. The chats are informal introductions to a charter educator commitment or commitments.  They explain what that commitment means for educators, and suggests one or two implications for teaching in a higher ed setting.

Here is a summary of the month each topic will be posted, its title,
and the related educator commitment.

April: Graduates with perspectives and approaches the world needs

  • Explicitly recognize their own position and work to understand, acknowledge, and value perspectives and worldviews different from their own

May: Building Broad Minds: Active learning strategies for large classrooms

  • Exemplify active learning and curiosity, demonstrate broad thinking

June: High quality, respectful classroom dialogue

  • Encourage and foster open and healthy dialogue
  • Engage with students and peers in a respectful manner
  • Co-create, with students, a shared space for learning in which all participants, including graduate and undergraduate teaching assistants, feel respected, valued and empowered to contribute as they achieve their goals and share the gifts of their identities in relationship with one another

July/August: You need to practice your research skills early and often

  • Provide opportunity for students to be inspired and engaged with and in the process of authentic inquiry, wherever possible, in their learning

September: All Aligned – Outcomes (3 part series on alignment)

  • Be aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity

November: All Aligned – Outcomes-based Assessment

  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning outcomes
  • Design tools to both assess and enable student learning
  • Learn about advances in effective pedagogies/andragogies

December: All Aligned – Instruction

  • Be aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity
  • Ensure that content is current, accurate, relevant to learning objectives, representative of the knowledge and skills being taught and appropriate to the position of the learning experience within a program of study

January:  Transparent Assessment

  • Provide a clear indication of what is expected of students in a course or learning activity, and what students can do to be successful in achieving the expected learning outcomes as defined in the course outline
  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning Outcomes
  • Design tools to both assess and enable student learning

February: Feeding Learning: Mark better work, in less time

  • Provide prompt and constructive feedback for students on their learning progress at regular intervals throughout the course

March: Reflection and Dialogue

  • Engage in meaningful conversations about their practices with others
  • Seek candid feedback from students about their learning experiences
  • Seek feedback from peers and other sources to allow for evidence on all aspects of teaching

Tamarind, Teaching and Undergraduate Research

Posted on by
Reply

[social_share/]



For the first time today, I tasted tamarind. I felt like I had discovered something so surprisingly delicious and interesting that I wondered why I had gone this long without knowing about it. This fruit’s benefits are wide-ranging and well known apparently—they just hadn’t been to me. I wasn’t introduced to it through family or friends, but I found information about it as I was searching for ways to reduce fluoride accumulations in the body—I was trying to solve a problem and it was one of the possible solutions to the problem.

New teaching strategies—or new-to-you teaching strategies—can be similar to discovering the tamarind fruit. Billions of people over the past 4000 years have been familiar with it, but it wasn’t within my realm of experience due to the limits of my family and social groups. With a problem to solve and Google, set off to find this fruit.

Now the leap to teaching and research…

Research I carried out on the integration of active learning into undergraduate classes, found that faculty integrated new strategies in response to problems they were trying to solve. They “found” a new strategy at a conference, workshop, or through journal articles and then used it to see if this new strategy or approach solved the problem. If it made a difference, the method was embedded in the course and if it didn’t make a difference it was dropped. Instructors used informal cycles of action research and reflective practice to renovate their teaching practices.

So here’s the thing…whether we recognize it or not we are doing “research” all the time in response to the problems we want to solve or to satisfy our curiosity. Research is learning, and learning is changing how we interact with the world. (You may have recognized this as a subtle plug for undergraduate research if you are a member of the pilot group 😉

The teaching problem: You might want to have students more engaged in their own learning because you know they learn better if they are engaged. You heard about cooperative learning being one of the most engaging teaching strategies. You find out how to implement cooperative learning and you give it a try. If you see students more engaged, you will use it again. If not, well, it was worth a try. Think tamarind fruit and be curious. And remember that just because a teaching strategy might be new to you, there are others who are very familiar with it because they have grown up in a culture where it is common practice. They can help you integrate a new strategy into your class. A good number of people who know about a variety of effective teaching strategies happen to be in the Gwenna Moss Centre. Give us a call if you are interested in more information about a wide variety of teaching approaches—including cooperative learning and undergraduate research

PhD Reform: A Speedier and Dissertation-Free Degree?

Posted on by
Reply

[social_share/]



Not long ago, I began the arduous process of applying to PhD programs. I didn’t make it far. What stopped me was not a lack of desire to push learning further, to what most graduate students see as the logical end of journey that began with their first university class. I was stopped by the nagging sense a PhD would simply take more time and resource than I had available.

Because I disliked falling prey to so utilitarian an impulse, I began looking into the PhD itself, to better understand why such a worthy intellectual endeavor appeared unsustainable and to find out if other students felt the same way.  My search led me to numerous blogs and reports about the PhD in today’s world, some of which can be found in my blog post about alt-ac careers. (alternative academic careers).

Wondering what to do with a PhD is, however, not the same as wondering why one would do a PhD at all. The latter question is better answered by examining the process rather than the outcome of earning the degree. The Academica Group’s Top Ten list featured a short round-up of current positions taken on the future of PhD programs, some of which were presented at a round table discussion during the 2014 Congress of the Canadian Federation for the Humanities and Social Sciences.

Two of the projects featured were McGill University’s White Paper on the Future of the PhD in the Humanities and the Modern Language Association’s Report of the Task Force on Doctoral Study in Modern Language and Literature. Both of these documents recommend extensive changes to the PhD, as well as investigating career outcomes.

Both documents recommend shortening the time to completion and increasing engagement with the world outside academia. To speed up the process and increase engagement, both explored the possibility of replacing the PhD dissertation with, for instance, “a coherent ensemble of scholarly projects,” recommended by the White Paper.

Simply speeding up the time to completion would certainly reduce the opportunity cost of a PhD program, but is this a realistic goal, even if the traditional dissertation is abandoned? Alicia Peaker, development editor at GradHacker responded in this interview, “what Graduate students need is not more or less time – it’s more support.” The debate continues and is worth following, particularly if you are a student currently looking at PhD programs.