Evaluating Teaching: Can We Get Beyond the Banality of SETs?

When I first began teaching in a tenure track position, my colleagues advised me that I had joined a department with a deep bench of knockout lecturers. One, I was told, could breeze through a detailed 50-minute lecture without once glancing at his notes (which he conspicuously placed on the lectern at the start of each class so as not to unsettle the students). The A-team’s lecturing prowess was reflected in the outstanding teaching evaluations they received (and is still vividly recounted by their 60- and 70-year old former students who return to campus for alumni weekend). If I wanted to garner equally stellar evaluations, it was impressed on me, I would have to step up my lecture game.

Economics Professor William H. Kiekhofer, lecturing in 1940 (Univ. of Wisconsin Archives).

I actually got pretty good at it – not to the level of memorization, but still good enough to boost my evaluations. All good, until it dawned on me that lectures could be inspirational, drawing students into the subject matter, but they weren’t the best way to promote student learning. And even the best lecture didn’t seem to capture the interest of those who purposefully occupied the back rows in my classes. My evaluations, while often good for my ego and, no doubt, helpful in my yearly evaluations, didn’t indicate whether any learning was taking place in class.

Encouraged by some reading in the scholarship of teaching and learning, greatly inspired by colleagues near and far, and (by then) protected by tenure, I moved away from lecturing in an effort to better address student learning. I had changed my classroom practice – but the evaluation process remained remarkably static, still using the same end-of-term questions to measure my effectiveness as a teacher.

Continue reading

Set for SETS? Student Evaluations of Teaching

Steve Volk, November 28, 2016

Among the relatively few rules that govern what we do in the classroom and how we do it  is the requirement that all teaching faculty hand out evaluation forms “near the end of each semester” (College) or “before the end of each semester” (Conservatory).  In the unstructured, devil-may-care past, each department (and each individual in the department) was pretty much free to design its own evaluation form, at least in the College, and I’ll just stick to Arts & Science here since the Conservatory has its own rules. That somewhat chaotic system, which made cross-departmental comparisons difficult since different attributes were measured and recorded on different scales ranging 3-point to a six-point scale, was put to rest some years ago. The current forms are designed around a standard one-to-five scale in six broad areas which the research has shown to produce (the most) valid and reliable results: 1) course organization and clarity, 2) instructor enthusiasm, 3) teacher-student interaction, rapport, and approachability, 4) workload and course difficulty, 5) assessments: exams, papers, grading fairness, and feedback, and 6) self-rated learning. We have standard rules about how they are to be distributed, collected, and returned to the faculty.

That said, there remains a lot of controversy about the value of such an exercise, not just among those who would argue that students shouldn’t be evaluating faculty at all (by my guess, a relatively small number) to those who think that the forms don’t actually tell us much about our teaching, to those who think that they don’t tell us anything about student learning – which is something we actually should be measuring – to those who argue that the research clearly demonstrates that SETs are significantly biased against many different subcategories of faculty:  women (female faculty in physics in particular), faculty of color, Asian faculty, international faculty who speak “accented” English, faculty who teach quantitative methods courses, and  “less physically attractive” faculty. Continue reading