Recipe for SoTL




Many a metaphor is used to make new ideas feel more familiar.

I’m an avid baker, so I wanted to share this alternative sweet way of seeing the elements and processes involved in the scholarship of teaching and learning (SoTL).  Let me know if you’ve got something cooking.

Book in a Loaf Pan

Photo by Brian Hoessler

1 cup questions, concerns or new possibilities

2 cups curiosity and excitement for your teaching and your students

1/2 cup reading literature inside your discipline about teaching courses and students like yours (see for example the list at http://pod.nku.edu/sotldisc.asp)

1/2 cup reading literature from educators in other disciplines with similar questions or approaches (see the list at http://www.issotl.org/SOTL.html)

4 tablespoons appropriately chosen methodology that makes sense for you

2 tablespoons relevant analysis

4 teaspoons of reflection as a practioner on your teaching and SoTL experiences

Instructions:

Blend questions and concerns with curiosity and excitement together until coalesces. Shift together readings and fold in. Add in methodology and analysis, and sprinkling in reflection. Stir thoroughly.

Pour into single large pan or into several smaller pans. Fill about 2/3 full initially as often expand during writing.

Bake through reflection, writing and revision.

Serve to audiences near and far.

 ~ from the kitchen of  Carolyn “Dr. Cupcake” Hoessler

Resources:

For a clearly written set of instructions see Donna M. Qualter’s “Six Steps for Turning Your Teaching into Scholarship

My favourite survey design resource, beyond Fluid Surveys (free for us at the UofS), is the  “Basics of Survey and Question Design” webpage

Also come by or drop us a line at the Gwenna Moss Centre for Teaching Effectiveness in the Murray Building right across from the Murray Library (and Starbucks) entrance.

Richness of Research on Active Learning: Let’s Stand on the Shoulders of Giants (or at least Other Educators)


Standing on the shoulders of giants
The National Survey of Student Engagement (NSSE) and scholarship of teaching and learning research promote the benefits of active learning, student engagement, and faculty-student interaction with courses that challenge students, shake them out of the passive listener role, and engage them in collaboration with peers to improve student academic performance. Prior evidence and strategies are summarized for several disciplinary areas including.

In addition to research studies in many disciplines including:

For a very comprehensive list of related articles see Ciaccia, Tsang, and Handelsman’s summary.

Highlighting the impact of short activities mixed with lectures, collaborative and cooperative activities, and an environment of interaction between students, these evidence-based approaches appear feasible and effective in improving academic performance.

When engaging in active learning, existing books and article gives hints about what to expect, tested strategies, and possible processes. Active learning challenges students to engage with materials and to deeply learn interconnections and potential applications. However, active learning needs to be explicitly explained to students who are often unaccustomed to such learning. Also, evaluations of your active learning course need to access student learning and other outcomes in ways that do not rely on course evaluations that may measure liking rather than learning.

Investigating the Scholarship of Teaching and Learning (SoTL) Landscape at the University of Saskatchewan



With Stan Yu

In the summer of 2012, the GMCTE, with support from the Office of the Vice-President Research, undertook a study to assess the level and extent to which the scholarship of teaching and learning, or SoTL, was being conducted at the University of Saskatchewan. SoTL can be defined as, “[the] systematic study of teaching and/or learning and the public sharing and review of such work through presentations, performance, or publications” (McKinney, 2006, p. 39). The study identified a campus-wide community of 284 SoTL scholars, consisting of 247 faculty and 37 staff members. An electronic survey was sent to this cohort, which yielded a 70% response rate.

In the electronic survey, we found that, for 47% of faculty respondents, SoTL comprises less than ¼ of their scholarly work. Meanwhile, 18% indicated that SoTL comprised more than ¾ of their scholarly research. While half of faculty respondents indicated that the proportion of their participation in SoTL has not changed over time, 41% now spend more time on SoTL than in the past. Additionally, 64% of staff conducted SoTL as part of their professional responsibilities. For 27% of faculty and 81% of staff, SoTL projects were conducted collaboratively more than half of the time. 49% of faculty and 41% of staff reported that their SoTL projects were multidisciplinary. Finally, it was found that a considerable amount of faculty (40%) and staff (26%) have published, and 53% of faculty and 50% of staff have presented their SoTL findings at a conference(s).

When asked to describe any barriers arising uniquely from involvement in this type of research, faculty respondents indicated that the lack of perceived legitimacy of SoTL scholarship constituted the primary barrier. SoTL work tends to be viewed as “soft” or “secondary,” and this point of view pervades everything from departmental cultures to promotion and tenure standards. For these faculty members, the work is carried out in spite of this friction.

Overall, this study revealed a sizable community of scholars internally networked across disciplines, departments, and colleges on campus. Furthermore, the quantity of U of S scholars engaged in, as well as their level of engagement with, SoTL is increasing. Despite the reported challenges of the perceived legitimacy and value of SoTL continuing to be pervasive and substantial, faculty and staff involved in SoTL research have had remarkable success. The next step towards furthering this type of scholarship is an effort to move SoTL beyond the practice of individuals towards the institutionalization of SoTL. For us, this elicits the work of Hutchings, Huber & Ciccone (2011) which documents strategies for institutionalizing SoTL, such as: articulating an integrated vision for SoTL and connecting SoTL to student learning initiatives. This snapshot of SoTL on our campus provides an optimistic and encouraging picture moving forward.

For a copy of the full report: http://www.usask.ca/gmcte/resources/library

Student Evaluations of Teaching: What are we really measuring?



With Sheryl Mills


On the recent CBC Early Edition podcast, the issue of what standardized testing was really assessing was raised. I find a similar concern arises with student evaluations of teaching. The debate of the validity and meaning is not new, but recent findings further suggest that when asking student about their instructors what we are actually measuring may not be what we expected. We may be looking at the gas gauge to measure speed.

Weer met de auto / Driving again
We do not appear to be measuring learning, or at least the actively engaged involvement with material that produces increased confidence, higher attendance, greater usefulness of reading textbooks, and better performance in their grades found in Walker and colleagues’ comparison of a traditional and an active learning section of introductory biology.

For unlike a car we are not measuring a physical property of speed or volume of gas in a single vehicle. Instead we are measuring the understood experiences of many individuals as they relate them to a particular question. We may be measuring their satisfaction with their grades as demonstrated in Tracy Vaillancourt’s recent three experimental studies where “students did seem to reward professors for good grades with high SETs and did seem to punish them for low grades (p. 10).

We are also capture on students’ memory tendencies to remember negative (or positive) moments, as well as their awareness and definitions of what we are asking about. For in order to report objectively that they found the course interesting, students need to define “interesting,” connect that definition to their experiences in the course, recall general patterns across the course, make a consistent assessment, and record that assessment carefully. What if interesting is a word associated with movies or favorite activities? And so on…

A very messy leap of faith indeed, yet one that can be complemented and framed to improve how accurately we are measuring what we hope to measure through multiple strategies including:

  • Peer evaluations
  • Focus groups with students that explore instances, definitions, and experiences
  • Surveys immediately after an activity rather than the end of term
  • Framing of activities to define rationale or purpose and then assessing specifically related to those goals rather than broad constructs.
  • Specific questions that provide or invite definition and examples from across timeframe, as well as focus reflection on specific details or at the very least on the breadth of things to start, stop, continue, positive, negative etc.

The ongoing debate suggests highly variable usefulness of student evaluations of teaching; there are many dials in the car and sometimes we pick the right one. Perhaps we should also look under the hood, drive at test distance, and hold up a radar speed tracker.

Self-Assessing Your Teaching



Kathryn SutherlandDr. Kathryn Sutherland, Associate Dean in the Faculty of Arts and Humanities at Victoria University Wellington, visited the U of S campus in November 2012 to speak about early career academic success, including at a talk she gave for the Gwenna Moss Centre.  In her talk she reminded me of something I read years earlier from Glassick et al’s (2004) Scholarship Assessed that is very important as we think about self-assessing our work as teachers.  Glassick et al (2004) propose the following criteria for evaluating all scholarly work:

1. Clear goals—Does the scholar state the basic purposes of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?

2. Adequate preparation—Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?

3. Appropriate methods—Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?

4. Significant results—Does the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?

5. Effective presentation—Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?

6. Reflective critique—Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?

It is quite easy to imagine how you might use these 6 criteria (and the sub-questions indicated by the authors) in assessing your research work – whether discovery research in your discipline, or the scholarship of teaching and learning (SoTL).  Indeed, as a member of a SoTL journal editorial board, and as the co-editor of the STLHE Green Guide series, these six criteria are used to inform how we peer review contributions to these forms of scholarly publication.  I have also used these six categories as the basis for a rubric for assessing students’ research papers.

In referencing this work, however, Kathryn was referring to using the framework to think about how we might assess and reflect on the quality of our teaching activities more generally.

There are many reasons for why Kathryn argues for engaging in reflection on your teaching, including documenting your effectiveness for applications to faculty positions (particularly for graduate students and postdoctoral fellows), for tenure and promotion case files, and for awards nominations. And it is tempting to spend time in this post exploring what others, including Schon (1983) and Brookfield (1995), say about reflective teaching practices.  But instead, let’s consider the six criteria in Scholarship Assessed as a tool for reflecting on, and self-assessing, your teaching.

  1. Clear goals—Have you set clear goals for what you are trying to achieve with your students’ learning (by defining learning outcomes for the course as a whole as well as for each individual class session) or personally with your teaching?
  2. Adequate preparation—Have you prepared yourself (in terms of both content knowledge as well as preparation for the teaching strategies you are using), your course materials (whether distributed to students or not), your assessments (exams, assignments, etc.), and your students (by setting and communicating clear and appropriate expectations)?
  3. Appropriate methods—Have you chosen the teaching strategies and assessment strategies that are MOST appropriate (in best alignment) with the intended learning outcomes you have established for your course or any particular class session?  How do you know whether you have done so?
  4. Significant results—Have your students (or at least a significant majority) achieved the outcomes you have set for them? Have some students exceeded your expectations?  Have you exceed theirs? Have I received the peer and student evaluations of teaching that I expect for what I feel I have accomplished in the course?
  5. Effective presentation—Are you choosing, and implementing effectively, appropriate forms of presentation for the context of your course? Are you presenting your ideas clearly? Is your material well organized? Are you using humour effectively?  How do you know you are effective?  Have you effectively shared or discussed the ‘significant results’ of your teaching with colleagues?
  6. Reflective critique— Are you critically evaluating your own teaching on a regular basis? Have you gathered appropriate evidence about your teaching to inform your reflection? Are you using that evidence to improve your teaching in the future?

If you would like to discuss reflecting on your teaching, and gathering evidence of your effectiveness as a teacher, do not hesitate to contact anyone at the GMCTE.  As well, if you are serious about improving your ability to document the effectiveness of your teaching then you might be interested in participating in the Teaching Portfolio short course offered annually as part of the GMCTE’s regular programming.  You can also find the full talk by Dr. Kathryn Sutherland online at the GMCTE website at: http://www.usask.ca/gmcte/events/featured_speakers

References (all three available in the GMCTE library):

Glassick, C. E., M. T. Huber, and G. I. Maeroff (1997). Scholarship assessed: Evaluation of the professoriate. San Francisco, CA: The Carnegie Foundation for the Advancement of Teaching and Jossey-Bass, Inc.

Brookfield, S. D. (1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass.

Schön, D. (1983). The Reflective Practitioner: How professionals think in action. London: Temple Smith.