How to Make an Effective Rubric

Good rubrics have three key advantages:

  • If you develop them, they help you align your assignment with your outcomes
  • They help you have similar marks for different students’ assignments of similar quality (inter-rater reliability), if you practice using them with other instructors or your TAs
  • They increase student understanding of the skills you want them to demonstrate and focus your students specifically on those skills

Although a good rubric is very helpful, they can be hard to develop.  This video describes why we use rubrics, common mistakes we make as we create them, and how to make a good one.

Interested in more? View a one hour session from Sue Brookhart on creating and using rubrics.

Transparent assessment

Assessment practice is shifting away from comparing students to each other, or grade derived professor’s experiences and preferences.  Increasing, it is focused on comparing students to a clear learning outcome or goal for the assessment that everyone in the class knows in advance. The process of clearly articulating that goal and what we consider good evidence of it is called “Transparent Assessment.” The goal of all transparent assessment is to ensure students understand what they are trying to achieve or learn, so they can be more effective partners in that learning. Our Learning Charter has three learning charter educator commitments related our assessment:

  • Provide a clear indication of what is expected of students in a course or learning activity, and what students can do to be successful in achieving the expected learning outcomes as defined in the course outline
  • Ensure that assessments of learning are transparent, applied consistently and are congruent with learning outcomes
  • Design tools to both assess and enable student learning

5 techniques to make your assessments more transparent:

  1. Clearly articulate the specific skills and knowledge you want to see students demonstrate right before they start learning each class.  While it is important to put learning outcomes or objectives on a syllabus, student need our help connecting those outcomes to specific learning they are about to do.
  2. Double check your alignment between what you teach, your outcomes, and your assessments.  Are there some parts of your assessment task that are unrelated to your outcomes? Are you testing things you haven’t taught, like specific ways of thinking or presentation skills? Is too much of the assessment focused in one area relative to the time you spent teaching it? Does the test or assignment use the same language you used when you taught?
  3. Share or co-construct assessment criteria before student start work on assessments. Discuss them overtly and compare them to models and samples, until you are confident students know what “good” looks like, and how to achieve it.  It might cost you time in class, but it will save you a lot more time marking, and you’ll mark better work.  Think your students understand?  Ask what they are trying to demonstrate when they do the assessment.  If they tell you the parts of the task (what) instead of the purpose of the task (why, how), the assessment is still not transparent to them.
  4. Use assessment tools, like checklists and rubrics, that a student can interpret without understanding what you are thinking.  If the categories on your rubric are ratings like “good” or “well-developed” a student still has to guess what you mean.  Substitute descriptions that include specifics like: “The argument is specific and illustrated through examples. The essay explains why the argument matters.”
  5. Use students are resources to increase transparency.  Have them try small examples of the main skill you are looking to see, and then give each other feedback using the criteria.  It will ensure they read the criteria, and cause them to ask about criteria or assessment processes they don’t understand. You’ll ensure students get more early feedback without increasing your marking load.

Increased transparency is about everyone in the class working together to have students learn as much as possible and demonstrate that learning as effectively as possible. For professors, it means fewer questions challenges of grades and marking better student work.  When done well, it results in student better understanding the learning goals and being more invested in them.

Learn more

 

Top Hat: How is it being used at the U of S?

The University of Saskatchewan has a continuing commitment to a technology-enhanced learning environment for students and in January 2016 acquired a campus-wide license for the Top Hat student response system. Top Hat is a software-based student response system, incorporating a “bring-your-own-device” solution, that is available at no direct cost to instructors and students. The primary goal of Top Hat is to enhance the teaching and learning experience for both instructors and students by bringing more engagement and interaction into traditional passive lecture-style learning approaches.

Who we are

We are a research team at the University of Saskatchewan who are interested in student response systems with a specific focus on Top Hat, their pedagogical effectiveness, and investigating the best teaching practices for these systems. Our team is organized under the Scholarship of Teaching and Learning (SoTL) cluster titled “Technology-Enhanced Learning: An Assessment of Student Response Systems in the University Classroom.”

  • Carleigh Brady, PhD, Instructor, Dept. of English
  • Soo Kim, PhD, B.Sc.PT, Professor, School of Rehabilitation Science
  • Landon Baillie, PhD, Professor, Dept. of Anatomy, Physiology and Pharmacology
  • Raymond Spiteri, PhD, Professor, Dept. of Computer Science
  • Neil Chilton, PhD, Professor, Dept. of Biology
  • Katherina Lebedeva, PhD, Instructor, Dept. of Anatomy, Physiology and Pharmacology

In March of 2018, we invited all individuals with a Top Hat instructor account at the University of Saskatchewan to participate in a survey about the use of Top Hat on campus and to share their experiences.

Results

 A total of 58 instructors responded to the survey. We found the majority of instructors using Top Hat at the University of Saskatchewan:

  • incorporate it in class to assess student concept understanding, test student recall, and share student perspectives (opinions, experience, and demographics)
  • use it for asking questions, creating discussions, and monitoring attendance
  • prefer “multiple choice question,” “word answer,” and “click on target” formats
  • think that the greatest advantages of Top Hat are: increased participation and engagement, student assessment, instant feedback from students, and the system’s ease of use/functionality
  • consider that Top Hat’s biggest disadvantages to be the time investment for software setting-up and grading, design issues, and technical issues (e.g. room connectivity)

In summary, we found that most instructors using Top Hat found it effective in facilitating a collaborative teaching and learning environment. Top Hat encourages students to participate actively during lectures by asking questions and polling student responses online. Despite some disadvantages, Top Hat is still preferred over clickers for its increased functionality (various question formats, interactive functions, and use of graphics), as well as its instant feedback and results polling.

However, further studies should be conducted to systematically evaluate the effect of Top Hat on student academic performance.

 Find more information