Transparent assessment

Assessment practice is shifting away from comparing students to each other, or grade derived professor’s experiences and preferences.  Increasing, it is focused on comparing students to a clear learning outcome or goal for the assessment that everyone in the class knows in advance. The process of clearly articulating that goal and what we consider good evidence of it is called “Transparent Assessment.” The goal of all transparent assessment is to ensure students understand what they are trying to achieve or learn, so they can be more effective partners in that learning. Our Learning Charter has three learning charter educator commitments related our assessment:

  • Provide a clear indication of what is expected of students in a course or learning activity, and what students can do to be successful in achieving the expected learning outcomes as defined in the course outline
  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning outcomes
  • Design tools to both assess and enable student learning

5 techniques to make your assessments more transparent:

  1. Clearly articulate the specific skills and knowledge you want to see students demonstrate right before they start learning each class.  While it is important to put learning outcomes or objectives on a syllabus, student need our help connecting those outcomes to specific learning they are about to do.
  2. Double check your alignment between what you teach, your outcomes, and your assessments.  Are there some parts of your assessment task that are unrelated to your outcomes? Are you testing things you haven’t taught, like specific ways of thinking or presentation skills? Is too much of the assessment focused in one area relative to the time you spent teaching it? Does the test or assignment use the same language you used when you taught?
  3. Share or co-construct assessment criteria before student start work on assessments. Discuss them overtly and compare them to models and samples, until you are confident students know what “good” looks like, and how to achieve it.  It might cost you time in class, but it will save you a lot more time marking, and you’ll mark better work.  Think your students understand?  Ask what they are trying to demonstrate when they do the assessment.  If they tell you the parts of the task (what) instead of the purpose of the task (why, how), the assessment is still not transparent to them.
  4. Use assessment tools, like checklists and rubrics, that a student can interpret without understanding what you are thinking.  If the categories on your rubric are ratings like “good” or “well-developed” a student still has to guess what you mean.  Substitute descriptions that include specifics like: “The argument is specific and illustrated through examples. The essay explains why the argument matters.”
  5. Use students are resources to increase transparency.  Have them try small examples of the main skill you are looking to see, and then give each other feedback using the criteria.  It will ensure they read the criteria, and cause them to ask about criteria or assessment processes they don’t understand. You’ll ensure students get more early feedback without increasing your marking load.

Increased transparency is about everyone in the class working together to have students learn as much as possible and demonstrate that learning as effectively as possible. For professors, it means fewer questions challenges of grades and marking better student work.  When done well, it results in student better understanding the learning goals and being more invested in them.

Learn more

 

All aligned – Instruction

In higher education, we have our students do all the hardest learning by themselves.  As academics, our greatest strength is expertise, but we routinely select passive instructional strategies that have our students mostly listening to lectures in our classes and doing their learning later.  Choosing passive listening robs us of the opportunity to provide the nuance and clarification that learners need while they learn. This post focuses on selecting the right type of instructional approaches to have our students actively learning the most important and challenging things they will need.

Relationship to our Learning Charter:There are two learning charter educator commitments related our instructional approaches to learning tasks:

  • Be aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity
  • Ensure that content is current, accurate, relevant to learning outcomes/objectives, representative of the knowledge and skills being taught and appropriate to the position of the learning experience within a program of study

Aligning the type of learning and your outcome: The type of learning you want your students to do dictates your instructional approach.  If the task is to recall factual information, but not be able to use it is any way, lecture is actually a very effective way to communicate that information.  Student will still need to rehearse it (memorize) by studying in order to learn, and sadly, will often forget much of it six months out.  In addition, the most useful things taught by an expert are rarely basic facts. They are skills, concepts, and refined understandings, which novice students learn most effectively while actively engaging in learning facilitated by an expert. When we intersperse passive teaching with the right type of active learning given our outcomes, students are much more likely to learn the most challenging things we have to teach.

Choosing the right strategy:

  1. Determine the type of learning you want students to do (not just the content you want to cover) by writing or using a good learning outcome.
  2. Select an appropriate active approach, and intermix it with your passive approaches to increase the amount of student learning.
Type of learning Instructional approach
Knowledge: factual information like terms, classifications, and theorists · Passive: Tell student about the knowledge (lecture, video, reading)

· Active: Have student use the facts in meaningful ways to learn them (mind-mapping, listing, drill and practice, sorting/drag and drop)

Conceptual: ideas understood well enough to apply it in new situations to assess or evaluate, like the concept of a successful argument or the concept of balanced · Passive: Read a complex explanation, hear someone describe the concept

·  Active: Classify or sort parts of the concept using criteria, refine an example of the concept, find errors, render judgement, construct an example of the concept, compare personal understanding to an example or rubric, reflect on growth of conceptual understanding over time

Process (cognitive): use a series of mental steps to accomplish a task, like solving for X

Process (psycho-motor or physical): use a series of physical steps with the right degree of acuity, like a neat set of the correct stitches

·Passive: observe someone do the steps

·Active: Try to do the steps, put the steps in order, find errors in someone else doing the steps, predict what will happen if the steps are done wrong, reflect on personal success in completing the steps

Skill: Combing multiple types of learning to accomplish a goal, for example identify the critical parts of a complex problem, choose the order to do it in, and solve the problem correctly · Passive: Hear about or see someone else using the skill

· Active: Try the skill in context (experiential learning) and reflect on success, complete a simulation, generate a decision-making tree or matrix, construct an argument on the implications of the application of the skill by someone else, provide feedback to another person by comparing their use of the skill to criteria

Learn more:

Read the other blogs in this sequence about constructive alignment:

Read the other chats related to Our Learning Charter to learn about other educator commitments.

Outcomes-based Assessment

Traditional forms of assessment, often norm-referenced, are increasingly mixed with outcomes-based assessment in campuses in Canada.  Often, outcomes-based systems start in professional programs with accreditation standards, where it is important that all graduate have minimal standards of competence, and are not just rated in comparison to their peers.  As the use of outcomes-based teaching and assessment is becoming more common, people are wondering what the difference between traditional and outcomes-based assessment is.

What is outcomes-based assessment?

  • It starts with faculty members articulating what they want students to be able to do when they complete the learning. This is called an outcome and it is different than thinking about what you will teach, because it is focused on the end result for students that you built the course for (learn more about outcomes and how to write them)
  • Outcomes-based assessment is the deliberate collection of evidence of student learning based on outcomes. It yields a mark relative to the outcomes (criterion referenced) rather than other students.

What do you do differently in an outcomes-based system?

  • Planning: You need to clearly know what level of skills and knowledge you will accept as competent or successful performance by students (instead of what mark), then plan the assessments to measure it and the instruction to teach it.
  • Assessment: You make an effort to give more feedback early, and give more attempts.  You pay more attention to most recent evidence. A good way of thinking about this is a driver’s test.  You don’t average in the first time you practiced with your final road test. Even if you fail your road test once, you get the score from your second attempt, not the combined score from your first and second, because the goal is to measure how competent you’ve become, rather than average in all attempts.
  • Calculating grades: In an outcomes-based system, you group and weigh by outcome, not by assessment task.  This means that outcome 1 might be 10% and outcome 2 could be 8% of the final grade. An individual test might have 50% of its marks in outcome 1 and another 50% in outcome 2.  In a syllabus, you’d explain your weighting by outcome, not by how much the assignments and final are worth.

Why would you bother to do this?

  • Students know exactly what you are trying to have them learn, and are more able to take responsibility for learning it.  As a result, they learn more deeply, and they more accurately identify what specifically they need to work harder on.
  • Because students understand more about what you want them to know and be able to do, you mark fewer weak assignments, which saves you time and frustration
  • It allows you to clearly understand how your students are improving over time on each of the outcomes you set.  That makes it easy to modify your course to make it more effective
  • Outcomes-based assessment is helpful for seeing how your course contributes to student success in the program, or gives a clear indication of how well student learn specific pre-requisite knowledge and skills you are trying to teach them

Why might you choose not do this?

Aside from the effort involved in trying something new, even if it might improve student learning, there are two common reasons to be concerned:

  1. The outcomes might be too focused on some government or institutional agenda, limiting your choice
  2. Students might focus on the outcomes instead of the learning

In response to #1, if you set your own outcomes, it is unlikely you would pick something you thought was unworthy or too low level, but there are times where an external body (like in accreditation) may set an outcome you would not have chosen to insure a progression of learning over courses and years. With regards to #2, I think we all want students to focus on the learning, but if they don’t know the outcome, they actually typically focus on the assessments, asking questions like “will this be on the test”, which indicate the are unclear on the learning and just focused on the grade.

What goals of the university is this connected to?

Outcomes-based assessment is directly connected to 3 key commitments we have as educators on campus, according to our Learning Charter:

  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning outcomes
  • Design tools to both assess and enable student learning
  • Learn about advances in effective pedagogies/andragogies

Read the other chats related to Our Learning Charter to learn about other educator commitments.

What works with study of teaching and learning?

The Gwenna Moss Centre for Teaching and Learning (GMCTL) offers grants for groups of faculty investigating what works in teaching and learning practice.  For most of, the process of teaching causes us to question, asking ourselves things like:

  • Why did that class work so well?
  • Why did students struggle so much with a particular question on a test?
  • What can I do to help with the increase in anxiety my students are reporting?

Gary Poole tells us is important to trust ourselves that these are great questions, and to work to explore them.  We need to consult the research in case there is already a well-established answer, we need to start small, and we need to work with others.

GMCTL offers clusters of faculty grants to help them do all three. However, many faculty still aren’t sure how to proceed with research into to teaching, specifically, so the process is still daunting. Dr. Nicola Simmons from Brock recently shared a set of videos that demystify the process, and this one with Gray Poole is a great simple introduction to what processes work in studying teaching and learning (SOTL).

If you are interested in learning more about studying your teaching practice, or want support working on a SoTL project with colleagues, contact GMCTL.

All Aligned – Outcomes

This post is one of a 3 part series on the concept of alignment of what you want students to learn, how you plan to teach them, and what you will assess them on.  Sometimes called constructive alignment, it has three parts:

  • Your learning outcomes
  • Your instructional approach or learning strategies
  • Your assessments

This post focuses on the need for clear learning outcomes for your students, and the next two posts in October and November focus on instruction and assessment respectively.

Why outcomes

Outcomes are statements that describe what our teaching is designed to help students know, do, or be. They start with a verb, then connect that to the key content.  Knowing what they are helps us design instruction that is focused exactly on what we want.

For example, if your goal is to have students

  • Articulate a well-defended argument based on precedent

Then you want to teach them to

  • Use criteria for well-defended to assess and build arguments
  • Find and make sense of relevant precedent

Without a clear outcome about being able to actually make a well-defended argument, you might think your instruction should stop at describing criteria for well-defended, or that a test where students recalled the definition might be better than an assessment where they actually had to make an argument.

Good outcomes help us focus our classes on the knowledge, skills, and values we actually care about our students learning, rather than explaining facts and ideas.  The educator commitments in Our Learning Charter describe being “aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity” as a key shared commitment of those of us who teach on campus.

More about outcomes

Read the other chats related to Our Learning Charter to learn about other educator commitments.

 

Practice Your Research Skills Early and Often

by Merle Massie, Coordinator, Undergraduate Research Initiative

We tend to think of university as a place to soak up knowledge, to learn stuff, to end up with a ‘brain full of smartness,’ as one twelve-year old boy once explained.

Yet the new University of Saskatchewan Learning Charter promises, and expects, more.

While content knowledge is important, the Charter sets out skills and practices that students, faculty and staff are expected to pursue.Three of those skills and practices are specific to research. USask wants students, faculty and staff to:

  • be able to locate, understand, evaluate and use information effectively, ethically, legally and with cultural appropriateness
  • develop and apply appropriate skills of research, inquiry and knowledge creation and translation
  • Communicate clearly, substantively and persuasively in different academic, professional and cultural contexts; nihtâ-âcimowin/nihta achimoohk (being a good storyteller).

An excellent place for students to start their skill journey is FYRE:
First Year Research Experience.

In 2018-2019, there were over 2700 student enrollments in FYRE classes. A FYRE class is a regular first-year 100 level course in every way except one: it deliberately uses research as a way to teach and learn.Students take on a research project (sometimes individually, sometimes in groups). The project must follow the research cycle: build a research question; investigate the question using the tools of the discipline; and share that knowledge with peers beyond the professor.

From the educator’s perspective, FYRE responds to another key point in the Learning Charter: strive for excellence in teaching. Using FYRE, the professor provides “opportunity for students to be inspired and engaged with and in the process of authentic inquiry, wherever possible, in their learning.”

A few thoughts on building FYRE into your first year course:

  • Consider course outcomes, beyond content. What skills does your discipline require? These could be discerning good sources, building and executing a survey, working in a lab or archive, analyzing data, synthesizing research to date, and so forth.
  • Build skill development into your course, scaffolding student learning. Give them room to develop skills, try, fail, and succeed.
  • Build a course component that could be a FYRE project. Students must move through all three parts of the research cycle: Question∼Investigate∼ Share.
  • Register your course with Merle Massie, Undergraduate Research Coordinator (email merle.massie@usask.ca) and choose a paid research coach.
  • Build a project timeline, with student feedback from a research coach and deadlines.
  • Design a way for students to share their findings beyond the professor.

FYRE allows educators to bring research into the classroom, to use it as a teaching tool, and to be up-front that FYRE is about learning and practicing research skills.

Research can be messy, confusing, frustrating and hard. It can also, at the same time, pull a student in and give them a whole new way to look at what it takes to build and add new knowledge, and to share that knowledge with others. Your learning, and theirs, will catch FYRE.

Learn more:

High quality, respectful classroom dialogue

This post is the third post in a series of the “Charter Chats” related to our new charter.  The others are linked on the bottom of this page. The chats are informal introductions to a charter educator commitment or commitments.  They explain what that commitment means for educators, and suggests one or two implications for teaching in a higher ed setting.

High quality, respectful classroom dialogue is essential in helping student learning.  When students are engaged in actively thinking about their own learning and discussing it with others, they are more likely to understand deeply. If students are just listening to an expert talking without the interaction, they are less likely to remember the learning 6 months later.  However, understanding more deeply and remembering more works best if the interaction in class is focused on the most important learning and it is safe and encouraged to share your ideas, even if they are not fully correct or are different. There are 2 key areas to pay attention to if your goal is improved learning through high quality, respectful conversation.

Open and healthy dialogue

The instructor is essential but not sufficient in creating open and healthy dialogue.  Good dialogue occurs when diverse perspectives are welcome in the group and genuinely considered, and that requires all group members. However, you can make a big difference to how likely good dialogue is.  Some key actions you can take are:

  • Share diverse perspectives and debates in disciplinary theory to make it clear good scholarship requires considering different ideas deeply. When possible, make space for plurality, rather than just one or two ways of thinking or being. We live in Treaty 6 territory and the homeland of the Metis, therefore including Indigenous informed perspectives and content along with Western ones is an essential first step in welcoming diversity
  • Describe what you mean by open and healthy dialogue before the first conversational opportunity starts.  In subsequent opportunities, discuss anything that needs “a refresher” to be a bit better than the last time. Focus on ways of communicating that are problematic, rather than describing specific students or ideas that are a problem
  • Have students work in smaller groups much more often than asking questions of the full class.  More students will get to participate, and most will feel safer to say more about what they think if they are only speaking to a few people
  • Overtly praise students who raise a potentially contentious activity delicately and effectively. Brave spaces are important
  • Warn students in advance if the content about to be discussed will be difficult or triggering, so they can manage their own emotions and expectations more effectively

How to create a shared, positive space in your class

Students know what it is to be positive in your class by what you do.  Your language, demeanor, and willingness to allow student to help shape the thinking and decision-making all help communicate what your classes will be about.

  • Use language that includes rather than excludes potentially marginalized groups
  • When dealing with a complex issue in class, ask yourself, “What story am I telling myself. What story might the other person be telling about themselves or about me?” Thinking it through helps with defusing, rather than escalating, potentially problematic moments
  • Co-create, with students, a shared space for learning through co-constructing or brainstorming together and working often in small groups as a part of daily class interactions
  • Raise errors as opportunities to learn.  Praise students who ask for clarification or surface their thinking about how to do something, even if it is wrong. Don’t pretend their processes are correct if they are not, but do point out that it was great to ask, and it is a very common misconception.
  • Don’t just share good examples.  Share examples with common errors and ask small groups to find the problems and explain why
  • Provide choice and voice anytime you can
  • Learn more about engaging with students and peers in a respectful manner with some tips for managing interaction in class, and use those strategies regularly
  • Ask students to set personal goals for the class, then allot time and marks to those goals
  • Use strategies like cooperative learning strategies or power sharing approaches like a talking circle to give student the opportunity to share the gifts of their identities in relationship with one another.  Seem to far from your disciplinary perspective?  Consider these 4 easy cooperative learning structures that are good across STEM and humanities courses

Learn more:

  • Attend Gwenna Moss Center for Teaching and Learning sessions on leading effective discussions or take our short courses on critical conversation
  • View other Charter Chats.

Building Broad Minds: Active learning strategies for large classrooms

Building broad minds is not about back filling.  Broad minds are the byproduct of encountering diverse ideas, thinking deeply about them, and integrating those ideas into our own worldviews and cognitive frameworks.  In higher education, the opportunity to be exposed to the thinking of a wide variety of disciplines usually happens at the first year level. However, those are also often large courses where the primary method of instruction is listening to your professor speak.  To actually get broad minds, our learning activities have to be active, even in the large classrooms where active learning strategies are limited by the room, and even when students are first encountering the subject mater.

A great simple rule for broad minds is the 10:2 ratio. It basically means that for every 10 minutes of lecture, and student needs 2 minute of social processing to make sense of it. Lots of the time, we think group work in classes is all about assignments.  Actually, it is much more about helping us make sense of what we learning. To encourage broad thinking, consider pausing at least every 10 minutes and doing a short activity that allows student to make sense of what you’ve just tried to teach them.

Use daily active collaboration in increase understanding

  • Having students problem solve in pairs
  • Having students turn and talk to each other about the implications of a new idea you introduced or why it matters
  • Provide opportunities to try something and get feedback from a peer. You don’t have time to provide all that feedback in a large class, but feedback is helpful in both improving and remembering, and there are many other people in the room who can help.  Giving feedback also refines our understanding, so your students are learning when giving and receiving feedback.
  • Have students teach each other something quick (not a big group presentation). Read more from the research about why this is one of the best ways to improve student understanding (Topping and Stewart, 1998).

Have students engage in critical thinking and think from multiple worldviews
Developing broad minds requires encountering the major debates of the discipline early and considering them from multiple perspective.  Although it is hard to have teams of two students debate other teams of two students in a large lecture theater, all students can think and speak simultaneously.  Want to see how it would work?

 

Learn more:

Graduates with perspectives and approaches the world needs

We often talk about the skills our graduates will need for success in their work and within our communities. As we aspire to be the university the world needs, we can’t overlook how essential perspective taking and cross-cultural competence are in our increasingly diverse world.  In this place, we have a collective commitment to improve the situation for the First Nation, Metis, and Inuit peoples, and to truth and reconciliation. And we can also see the impacts of nationalism and nativism on the global stage, A problem that is prompting us to equip our students with the skills they will need to respond.This post is one in a series related to the educator commitments in Our Learning Charter. It focuses on how to help students to explicitly recognize their own position and work to understand, acknowledge, and value perspectives and worldviews different from their own.

What you can model in your teaching:

  1. Start by acknowledging your own position and privilege. Being a role model and ally is essential in supporting students in the process of doing the same thing. Knowing why you would include a land acknowledgement, for example, rather than omitting one or just adding one to your syllabus is part of an acknowledgement. Not quite sure how to approach it in a good way? Join the short course in the fall on Indigenization, decolonization and reconciliation at the Gwenna Moss Center for Teaching and Learning (GMCTL).
  2. Purposefully include content, perspectives, and worldview from local Indigenous communities and international perspectives.  The focus should be on being prepared to support a diverse world and set of different views. Need some support?  Ask for a consultation at GMCTL.
  3. Deliberately offer more than one perspective on the debates of your discipline whenever possible, and explain the value of those discussions to the disciplinary discourse. Provide opportunities for students to engage in facilitated discussions about those debates without taking a position yourself.

What you could do with your students:

  1. Choose to share your own power by using active learning strategies to get students thinking and talking, rather than transmission styles where students mostly listen. Understanding,  acknowledging, and valuing perspectives and worldviews different from their own is requires active learning processes, because it requires students to be in dialogue with the other.  Learn more about the research on active learning or experiment with some active learning strategies in your class.
  2. Provide students with deliberate opportunities to work in culturally diverse groups where they’ll be exposed to a multiplicity of perspectives that they might not encounter, given that we are more likely to self-select groups of people like us.
  3. Proactively plan for how to have challenging conversations with students in class, and how to respond when students struggle to value worldviews and perspectives other than their own.  Not sure how do this? Join the GMCTL for a workshop series on planning for and responding to difficult conversations in the classroom or preview some online resources.

View other posts in the Charter Chat series.

Charter Chats

The University of Saskatchewan has a new Learning Charter.  First written in 2010, then updated in 2018, our charter is helpful in framing what we believe about teaching and learning.  It will also provide a sense of direction for work at Gwenna Moss over the next year:

The Learning Charter thus acts as a conceptual map and planning document, linking together our pursuits and how we strive for them, encouraging and guiding us on our educational journey. As a map, it is also a focal point for our community to discuss where we are and where we want to go in our shared future.

This post will be the hub of the “Charter Chats” related to our new charter.  As each new monthly post is written, the post will be linked on this page. The chats are informal introductions to a charter educator commitment or commitments.  They explain what that commitment means for educators, and suggests one or two implications for teaching in a higher ed setting.

Here is a summary of the month each topic will be posted, its title,
and the related educator commitment.

April: Graduates with perspectives and approaches the world needs

  • Explicitly recognize their own position and work to understand, acknowledge, and value perspectives and worldviews different from their own

May: Building Broad Minds: Active learning strategies for large classrooms

  • Exemplify active learning and curiosity, demonstrate broad thinking

June: High quality, respectful classroom dialogue

  • Encourage and foster open and healthy dialogue
  • Engage with students and peers in a respectful manner
  • Co-create, with students, a shared space for learning in which all participants, including graduate and undergraduate teaching assistants, feel respected, valued and empowered to contribute as they achieve their goals and share the gifts of their identities in relationship with one another

July/August: You need to practice your research skills early and often

  • Provide opportunity for students to be inspired and engaged with and in the process of authentic inquiry, wherever possible, in their learning

September: All Aligned – Outcomes (3 part series on alignment)

  • Be aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity

November: All Aligned – Outcomes-based Assessment

  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning outcomes
  • Design tools to both assess and enable student learning
  • Learn about advances in effective pedagogies/andragogies

December: All Aligned – Instruction

  • Be aware of the range of instructional methods and assessment strategies, and select and utilize teaching methods that are effective in helping students achieve the learning outcome of a course or learning activity
  • Ensure that content is current, accurate, relevant to learning objectives, representative of the knowledge and skills being taught and appropriate to the position of the learning experience within a program of study

January:  Transparent Assessment

  • Provide a clear indication of what is expected of students in a course or learning activity, and what students can do to be successful in achieving the expected learning outcomes as defined in the course outline
  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning Outcomes
  • Design tools to both assess and enable student learning

February: Feeding Learning: Mark better work, in less time

  • Provide prompt and constructive feedback for students on their learning progress at regular intervals throughout the course

March: Reflection and Dialogue

  • Engage in meaningful conversations about their practices with others
  • Seek candid feedback from students about their learning experiences
  • Seek feedback from peers and other sources to allow for evidence on all aspects of teaching

Plugin by Social Author Bio