In early June, the Bok Center hosted a talk by Dr. Samuel T. Moulton, Director of Educational Research and Assessment at Harvard University and part of the team implementing the Harvard Initiative for Learning and Teaching.
Dr. Moulton reviewed the research (more than 4000 articles!) that underlies the design and implementation of student ratings systems. Research on student ratings mainly investigates two outcomes of typical ratings surveys: reliability and validity. Reliability is the measure of agreement between different iterations of any experiment or survey, not just in one instance but also over time. Validity measures the degree to which the assessment captures the data it sets out to capture.
According to Dr. Moulton, studies have shown that student ratings tend to be very reliable and sometimes valid. In other words, those who claim that existing student ratings systems are worthless have ignored a great deal of careful research. Yet because student ratings sometimes score low on tests of validity, Dr. Moulton argued that researchers should continue to investigate other measures of teacher effectiveness and student learning, such as studies of changing attitudes and interests among students, or evaluation systems that do not rely so heavily on self-reporting.
Dr. Moulton also addressed persistent myths surround student ratings: that they are popularity-based (they aren’t), that students only appreciate good teaching well after they evaluate it (they don’t), and that feedback doesn’t affect teaching (it does).
Thanks to Bok TV, you can watch the entire talk below. I highly recommend it!