Aligning assessment and experiential learning

I didn’t know what to expect as I rode the elevator up the Arts tower to interview for a research assistant position for a SOTL group. I certainly didn’t expect the wave of information and Dr. McBeth’s joyful energy. She, Harold Bull, and Sandy Bonny explained the project in a unique dialect; a mix of English and their shared academic speak. I hope they didn’t catch onto my confusion when they were throwing around the term MCQ, or multiple choice question, (which refers to the Medical Council exam in my former profession). I realized that I had quite a lot to learn if I was going to succeed in this position. I’d need to learn their language.

The scholarship of teaching and learning cluster working group shared the project through a concept map that linked “Assessment” to different kinds of students, subjects, and teaching strategies. The concept map itself was overwhelming at first but organizing information is in my skill set so creating an index was a straightforward matter. Connecting that index to resources with EndNote was quite a different affair. I closed the program angrily after multiple failed attempts to format the citations in the way I wanted. I had listened carefully and taken notes with Dr. McBeth, but the gap between theory and practice was large. With some perseverance, I am now able to bend the program to my will. It is a useful tool but like a large table saw or pliers used to pull teeth, it still frightens me.

Working with the SOTL concept map, I had identified the three areas, and their sub-topics, which the group is most interested in exploring:

  1. Examination/Assessment
    1. Ease of grading
    2. After experiential learning
    3. Multiple choice questions (MCQ)
  2. Type of Experience
    1. Designed
    2. Emergent
    3. Prompted reflection and relativistic reasoning
  3. Subject Permeability
    1. Alignment to common knowledge
    2. Access to affordances for self-teaching and tangential learning

Well I might as well move my things to the library and stay there until May. These topics cover a huge swath of pedagogical research. As I began reading, though, I soon saw that there were emerging patterns and overlaps among topics. Designed experiences overlapped with assessments. Multiple choice questions and cognition intersected. It was clear that while my index was neatly laid out in discreet cells in Microsoft Excel, the reality of the discourse was a lot more fluid and messier; more accurately reflected in the hand-written topic names, lines, and stickers of the concept map.

An interesting thing I discovered was that although I struggled at times in my methodology class in Term 1, the information and skills I learned there were useful in evaluating sources. I can ask questions and identify gaps where methodological choices aren’t outlined clearly. To be able to use my skills in a practical manner immediately after acquiring them is very exciting.

“…student views and assumptions about experiential learning and peer assessment may not align with data on actual learning.”

Currently I am focused on the topic of Examination/Assessment, which has the broadest scope of all topics identified. Two articles about student perception of experiential learning and peer assessment were intriguing to me. They make clear that student views and assumptions about experiential learning and peer assessment may not align with data on actual learning. This resonates with all the learning I’ve been doing about subjectivity/objectivity and research ethics. Our perceptions and assumptions can be very powerful but they shouldn’t be taken as dominant knowledge without further inquiry.

Some authors make strong claims about their findings even though the description of their methodological processes is lacking. Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012) assert that their findings “vindicate multiple-choice tests, at least of charges regarding their use as practice tests” (p. 1342). I am hesitant to completely agree with their claim based on this article alone because certain methodological details aren’t addressed, such as participants’ demographics and level of study. They also changed variables (feedback and timing for answering questions – those without feedback got more time to answer) in Experiment 2 and used a pool of participants from another area of the county (United States). The work of Gilbert, B. L., Banks, J., Houser, J. H. W., Rhodes, S. J., & Lees, N. D. (2014) is also lacking discussion of certain design choices such as appending their interview and questionnaire items and explicating the level of supervision and mentorship (and any variation thereof) that different students received in different settings. This doesn’t necessarily mean that the authors didn’t make very careful and thoughtful choices, but that either the description of their research needs to be amended or further study is necessary before making definitive claims.

Conversely, the work of VanShenkhof, M., Houseworth, M., McCord, M., & Lannin, J. (2018) on peer evaluation and Wilson, J. R., Yates, T. T., & Purton, K. (2018) on student perception of experiential learning assessment were both very detailed in their description of their research design and the methodological choices made.

I wonder if the variability in data presentation is reflective of the varying skills of researchers as writers. Perhaps it is more reflective of the struggle of researchers to move toward an evidence-based practice in the scholarship of teaching and learning. Maybe it is both.

While I will not be creating a nest in the library and making it my primary residence, there is still a lot to read, learn, and uncover. I look forward to journeying together with you.

Summary

· Sources must be carefully evaluated to ensure quality of research design and findings.· Delayed elaborate feedback produced a “small but significant improvement in learning in medical students” [Levant, B., Zuckert, W., & Peolo, A., (2018) p. 1002].

· Well-designed multiple-choice practice tests with detailed feedback may facilitate recall of information pertaining to incorrect alternatives, as well as correct answers [Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012)]

VanShenkhof, M., Houseworth, M., McCord, M., & Lannin, J. (2018) have created an initial perception of peer assessment (PPA) tool for researchers who are interested in studying peer assessment in experiential learning courses. They found that positive and rich peer assessment likely occurs in certain situations:

  • With heterogeneous groups
  • In a positive learning culture (created within groups and by the instructor)
  • Clear instructions and peer assessment methodology

Wilson, J. R., Yates, T. T., & Purton, K. (2018) found:

  • “Student understanding is not necessarily aligned with student engagement, depending on choice of assessment” – Journaling seemed best at demonstrating understanding but had a low engagement score by students, (p. 15).
  • Students seemed to prefer collaborative assessments – seen as having more learning value in addition to being more engaging, (p. 14).
  • This pilot indicates that student discomfort doesn’t necessarily have a negative impact on learning, (pp. 14-15)

This is the first in a series of blog posts by Lindsay Tarnowetzki. Their research assistantship is funded by and reports to the Scholarship of Teaching & Learning Aligning assessment and experiential learning cluster at USask.

Lindsay Tarnowetzki is a PhD student in the College of Education. They completed their Master’s degree at Concordia University in Communication (Media) Studies and Undergraduate degree in English at the University of Saskatchewan. They worked at the Clinical Learning Resource Centre at the University of Saskatchewan for three years as a Simulated Patient Educator. They are interested in narrative and as it relates to social power structures. Lindsay shares a home with their brother and one spoiled cat named Peachy Keen.

 

Internationalization of Teaching & Learning : Featured Instructor

Photo provided by Dr Lucy R. Hinnie

Dr Lucy R. Hinnie
Postdoctoral Fellow

Lucy is a postdoctoral fellow in the department of English and completed her PhD at the University of Edinburgh, Scotland. In her work, she looks at written text through the frame of intersectionality, interrogating the accepted ‘canon’ of white male scholars and looking to find relevance to every student, regardless of their background.

She has a desire to strengthen her teaching practice and do better by all of her students.

She took the internationalization short course because she has a desire to strengthen her teaching practice and do better by all of her students, especially those who face difficulties in what is perceived to be standard classroom situations. For her, successful internationalization will look like an enthusiastic and multicultural student body who can engage in safe learning spaces with cultural sensitivity and awareness.

Connect with Lucy to learn more: lucyrhinnie.co.uk

Internationalization of Teaching and Learning : Featured Instructor

Jocelyn Peltier-Huntley, M. Sc., P. Eng.

Photo provided by Jocelyn Peltier-Huntley

Lecturer, College of Engineering

Jocelyn is a professional mechanical engineer. Her research is around understanding the gender gap in the Canadian mining industry. At a personal level, she wants to see positive change happen to move towards equity within our society. As an instructor of engineering design and communications, and as a professional, she feels it is vitally important to know how to understand and work with stakeholders who may be from a variety of backgrounds and have different ways of knowing.

Successful internationalization allows for all people to be fully included and empowered…

She took the internationalization short course to improve her teaching practices and also to help inform how she frames the messaging of her research on gender equity. Jocelyn believes that successful internationalization allows for all people to be fully included and empowered to fully participate and achieve their full potential in their education, careers, and lives.

Connect with Jocelyn to learn more: https://engineering.usask.ca/people/sopd/Peltier-Huntley,Jocelyn.php

How do I internationalize my course?

Self-reflection

Step 1: Know my position and privilege. Who am I as a teacher? (This idea isn’t new, check out this article from 1958: Teacher, Know Thyself)

Step 2: Does the way I design my course plan for access and diversity?

Step 3: Do I want to “add-on”, “infuse”, or “transform” my course through internationalization?

Some direction

If you are working on step 3, there is an excellent resource of teaching tips here: Strategies for Course Internationalization. Centre for Teaching Excellence, University of Waterloo.

A simple way to start internationalization is to add assigned readings from international perspectives. This can be a way to start conversations and look for similarities and differences in findings. Even the writing and presentation structure might reflect cultural differences.

Next, take a look at your course outcomes – are students expected to develop or use intercultural competencies? How might the next version of your course highlight internationalized or global community skills?

Onwards on this journey, it’s time to look at evaluation. Inclusive assessment should include students using a metacognitive process to track their development. If that sentence doesn’t make sense on first reading, try this: a student needs to be able to know what they know and how they know it at any stage of learning. If they are just beginning, they should be able to identify that, recognize when they are building knowledge/skills/attitudes, and ultimately know when they’ve mastered or achieved the outcome of the learning. When students are involved in the assessment process, they are demonstrating choice, responsibility, and reflection. These are all attributes of inclusive learning which is fundamental to internationalization.

Here is another list of tips and tricks to start internationalizing your course.

This post is part of a series in internationalization. You can follow along here.

Come say hi! We’re at the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops to meet your needs.

 

From Modelling to Designing Intercultural Curricula

You now know that you have pretty decent intercultural teaching capacities.

You have continued to develop an awareness of your own identity and are modelling perspective-taking. Students in your course have the opportunity to interact with different worldviews because you know that makes them smarter. You actively create opportunities to build relationships between ‘others’ and can recognize barriers to student participation – you’ve essentially mastered using your intercultural capacity to inform teaching practices. So now you must be wondering, “What’s next? How can I further internationalize in my course?”  No fear, you are not alone. Dimitrov & Haque (2016) have some suggestions for “curriculum design competencies”.

“Effective instructors are able to critically evaluate the curriculum and create learning materials that transcend the limitations of monocultural disciplinary paradigms, scaffold student learning so students have a chance to master intercultural skills relevant to their discipline, and design assessments that allow students to demonstrate learning in a variety of ways.” – Dimitrov, N., & Haque, A. (2016). Intercultural teaching competence: A multi-disciplinary model for instructor reflection. Intercultural Education, 27(5), 437–456. https://doi.org/10.1080/14675986.2016.1240502

Key questions to ask yourself on your internationalization journey:

  • Does my course syllabus have a specific learning outcome where a student is asked to demonstrate specific knowledge, skills, or attitudes of a global or international design?
  • Do all the authors of my selected articles look or sound like me and if so, why – and can I change this?
  • Are students asked to take different perspectives in assessed work (work that is evaluated for marks)?
  • Do students have any choice in their assessment? Are different communication styles encouraged?
  • Does my course allow students the opportunity to develop a more robust disciplinary identity aligned with their cultural or personal identity?

If answering these questions leaves you with more questions, it’s likely a good time for a conversation with the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops to meet your needs.

How Might Intercultural Capacity inform our Teaching?

Once we develop the capacity for intercultural competence, we can start to infuse the associated skills into our teaching practice. This can take many forms but all the elements connect to the group of knowledge and skills we associate with facilitation.  Pedogogy is the study of leading learners and facilitators make a process easier. So, facilitation is the process we use to make the learning possible. In adult education, we know that our learners come with valuable prior knowledge, skill, and experience. We can draw on these to enhance the learning experience for both instructor-facilitator and student.

How do we start facilitating?

As an instructor, or facilitator, you may wish to try some of these summarized strategies suggested by Dimitrov & Haque (2017) :

When can we use it?

Using your intercultural capacity can happen any time you are interacting with someone else. Everyone has inherent uniqueness that together creates diversity.  As our ‘village’ grows, our diversity along with it.

“…more and more of us do not live in closed circles of like-minded, similarly raised people. Think of the last few gathering you attended – a work meeting, a class, a trade show. Chances are, you sat next to and talked with people from places other than where you’re from, people with different cultural norms, people of different races and religions and histories. And chances are, therefore, that you sat next to people who do practice etiquette – but etiquette different from yours, and perhaps in conflict with it on certain points.”

 Parker, Priya (2018). The art of gathering: How we meet and why it matters. New York : Riverhead Books.

Why do we care?

Figure 1 Depiction of instructor trying to introduce too many new strategies at once. Illustration by author.

We care about using good facilitation because we want our learners to achieve the desired outcomes of the course in the most efficient and effective way possible. This means using strategies we know will allow learners to thrive. As instructors, we want to leverage the learner’s pre-existing knowledge, skills, and attitudes to make the new learning accessible and within reach. If the learning curve is too steep, learners may just fall off. And even for the instructor/facilitator, try to keep a growth mindset that we are all working to get along and want to be successful in our relationships. A positive disposition and honesty about one’s own positionality and areas for growth will go a long way!

If you’re looking for more help with developing your intercultural capacity, please reach out to the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops or short courses that can meet your needs.

NB: If you tried the text verification link above, hemingwayapp.com, you may be interested to know that this article is at a grade 10 reading level and 9 of the 39 sentences are “very hard to read”. Ideally, public text should be at a grade 9 level 🙂

What is Intercultural Teaching Capacity?

Intercultural teaching capacity is the ability to engage and support learners through difference in learning activities. Instructors who have intercultural capacity are able to bridge difference in the classroom and foster meaningful relationships with and among students. This is also called intercultural competency, but we like to think of it as a continual process instead of a checkbox. Instead of being just ‘competent’, we need the capacity to work with intercultural relations effectively, and those skills change and grow with each experience as we nurture them. Hence we can be capable or have the capacity to be interculturally competent.

Reasons why instructors may wish to develop their intercultural capacity is so that they may be more aware of their own power and privilege, their inherent biases, and to help them untangle the ‘hidden curriculum’ of attitudes they are inadvertently teaching alongside their content (knowledge and skills). Making the implicit, explicit, serves all learners and leads to a more inclusive and welcoming environment for learning. This aligns with the Learning Charter pursuits of truth and understanding, integrity and respect, and community pursuits. Being open and flexible to change and recognizing the value in other’s perspectives helps build meaningful and authentic community relationships.

Some questions to ask yourself if you want to reach all students so that they all have equitable access to success in your course:

  • For whom was the course curriculum designed? Who was the imagined ‘student’?
  • How does the curriculum reflect (mirror) the learners in the room, and also provide ‘windows’ to other perspectives?
  • How might we build a learning community in the classroom where learners draw from their own and others’ knowledge sources?
  • What is the difference between (and purpose of) assimilation and integration in the classroom?

 Three things you can do to build your intercultural teacher capacity might be:

  • model and encourage perspective-taking and openness to diverse ways of knowing;
  • facilitate dialogue about global issues using respectful, inclusive, and culturally relevant teaching strategies;
  • think abouy how  your assessments and curriculum design promote multiple perspectives among students.

Learn more:

McIntosh, P. (1998).  White privilege: Unpacking the invisible knapsack.  In S. Rothenburg, (Ed.) Race, Class and Gender in the United States: An Integrated Study.  New York: St. Martin’s Press.  https://files.eric.ed.gov/fulltext/ED355141.pdf?utm#page=43

Questions adapted from the original created by Shannon Morreira and Kathy Luckett, University of Cape Town.  Retrieved on July 30, 2019 from: https://folukeafrica.com/questions-academics-can-ask-to-decolonise-their-classrooms/

Come say hi! We’re at the Gwenna Moss Centre for Teaching and Learning. We can help individually or direct you to one of our workshops to meet your needs.

 

 

Plugin by Social Author Bio