An Evaluation of Course Evaluations

Journal Club Article: Stark, P. & Freishtat, R. (2014). An Evaluation of Course Evaluations. University of California, Berkeley.

The magic of evaluation, every nurse educators way of measuring teaching effectiveness. Or is it?

This discussion paper by Stark & Freishtat (2014) provides a great balanced discussion on evaluations. From my experience as an educator in the hospital and university setting, evaluation is deemed important in measuring the quality of education.

“Among faculty, student evaluations of teaching (SET) are a source of pride and satisfaction- and frustration and anxiety.”  (pg. 2)

That tenure and promotion are related to performance in evaluations, the pressure is on to make sure your students provide a positive evaluation. Imagine the pressure this potentially places on educators to please students, what about holding to account or daring to fail a student, imagine the bad evaluation you may receive. This provides a different spin to the magic of evaluations.

“students are in a good position to evaluate some aspects of training, but SET are at best tenuously connected to teaching effectiveness.”  (pg. 3)

Measurement of a rating scale of “effective to not effective at all” is easy, takes little time and resources. Without doubt, student feedback is an essential part of course evaluation but more thorough and in-depth approaches could be utilised, but they require more resources.

Can one educator really be effective for the mix of students across age, sex, background, learning style and abilities?

“there’s no reason nonresponders should be like responders- and good reasons they might not be. For instance, anger motivates people to action more than satisfaction does.”   (pg. 4)

Statistics and SET’s:

  • What is the minimum response rate required to be classified as ‘representative’ for further analysis?
  • Is using these statistics to create averages to compare to past courses, against departmental averages or wider organisation data meaningful?
  • Do we really consider these comparisons as safe assumptions of quality learning.

“What is effective teaching? One definition is that an effective teacher is skillful at creating conditions conducive to learning. Some learning happens no matter what the instructor does. Some students do not learn much no matter what the instructor does. How can we tell how much the instructor helped or hindered?  (pg. 10)

Stark & Freishtat highlight that measuring learning is extremely hard, and that currently we measure what students say, not actual teacher effectiveness, and pretend it’s all the same. Of course, all these arguments are also the same for using course grades for measuring quality, so what’s left? What about observation and providing feedback to educators on the teaching itself over a period of time, then this can align with the SET data to provide a bigger picture overview.

To Counteract: The Data Argument

Without DataReference

Stark, P. & Freishtat, R. (2014). An Evaluation of Course Evaluations. University of California, Berkeley.