The Science of Student Ratings

In early June, the Bok Center hosted a talk by Dr. Samuel T. Moulton, Director of Educational Research and Assessment at Harvard University and part of the team implementing the Harvard Initiative for Learning and Teaching.

Dr. Moulton reviewed the research (more than 4000 articles!) that underlies the design and implementation of student ratings systems. Research on student ratings mainly investigates two outcomes of typical ratings surveys: reliability and validity. Reliability is the measure of agreement between different iterations of any experiment or survey, not just in one instance but also over time. Validity measures the degree to which the assessment captures the data it sets out to capture.

According to Dr. Moulton, studies have shown that student ratings tend to be very reliable and sometimes valid. In other words, those who claim that existing student ratings systems are worthless have ignored a great deal of careful research. Yet because student ratings sometimes score low on tests of validity, Dr. Moulton argued that researchers should continue to investigate other measures of teacher effectiveness and student learning, such as studies of changing attitudes and interests among students, or evaluation systems that do not rely so heavily on self-reporting.

Dr. Moulton also addressed persistent myths surround student ratings: that they are popularity-based (they aren’t), that students only appreciate good teaching well after they evaluate it (they don’t), and that feedback doesn’t affect teaching (it does).

Thanks to Bok TV, you can watch the entire talk below. I highly recommend it!


5 thoughts on “The Science of Student Ratings

  1. I’m sure that by and large student ratings systems are accurate and the ‘myths’ are just that. However, I don’t recall having too many opportunities to provide my instructors with constructive criticism, simply because I felt it would have negatively impacted my student ratings (grades). Dr. Moulton’s work substantiates what most of us knew when we were in high school and college. Good instructors motivate us and poor instructors put us to sleep. Some ambitious students will excel no matter what, but the majority of us need some kind of inspiration.

  2. I’m curious why the 70s triggered the “explosion of this research?” and it seems to have topped out back then (around the 2:50 min marker in the video). I would’ve thought now would be the pace-setter for this type of study and followup. Either way its interesting.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s