Open Letter to Fr. Jenkins: Evaluating the TCEs
Observer Viewpoint | Thursday, September 15, 2005
In this final installment we turn to the issue of teacher-course evaluation. This issue is an offshoot of what appears to be the prevailing tendency among administrators to run the University like a business, as were the previously treated issues of public image and grade inflation.
I have my car serviced at a garage where they say my business is appreciated. After each visit I receive a questionnaire asking for evaluation of the service and for suggestions about how it could be improved. This questionnaire always makes me think of the TCE forms filled out by my students at the end of the term. These forms invite the students to evaluate my teaching performance, and to identify ways in which it needs improvement. The students have hired me to perform a service, and this is their chance to let people know whether they consider their money well spent.
There is something anomalous about the timing of these evaluations. College teaching is comparable to the practice of medicine, in that both are dedicated to the service of other people. Consider the case of someone who has just undergone major surgery. If the operation were immediately followed up by a questionnaire asking whether the doctor was attentive, the nurses cheerful and good service rendered overall, the patient might understandably throw it in the wastebasket. Only at the end of the recovery period is it appropriate to evaluate the success of the operation.
In the medical case, it is obvious that the effectiveness of the operation cannot be assessed by asking how the patient feels about it immediately afterwards. For similar reasons, it seems inappropriate to judge the effectiveness of a course by asking how the student feels about it before the term is officially over. The skills learned by the student cannot be realistically tested until put to use in the months and even years ahead.
There is no reasonable doubt that teachers should be held accountable for their classroom performance. Possession of an advanced degree is no guarantee that a person can teach, and those unable to do so should not occupy a teaching position. The University should strive for excellence in teaching, as in other academic matters. The question is how quality of teaching can best be evaluated.
Let us look at the strengths and weaknesses of our current system of TCEs. The present TCEs are useful in rating the instructor’s enthusiasm, ability to stimulate students and clarity of presentation. Such attributes are obviously important to effective teaching. The TCEs are also useful in pointing out problems like impatience and partiality on the part of the teacher, along with perceived inaccessibility outside the classroom. Problems like these can hinder successful teaching. For beginning teachers, it can be particularly helpful to address shortcomings of this sort before they become habitual.
A major disadvantage of our TCEs, on the other hand, is that they provide little information about what students actually learned in the course. In particular, they tell us very little about what the course has added to the student’s grasp of the discipline, or about the extent to which he or she has comprehended the relevant material. In this regard, it should be axiomatic that a successful course is one that advances understanding of the discipline on the part of its qualified students.
The underlying defect of our current system of course evaluation, it seems to me, is that TCE scores are strongly influenced by factors extraneous to teaching effectiveness. Foremost among such factors is student anticipation of grades to be assigned at the end of the course. Studies have shown time and again that a student expecting a top grade will rate the instructor highly on the TCE, with the opposite effect when a poor grade is expected. A research report issued a few years ago at Notre Dame, for instance, stated that “expected course grade is very strongly … associated with student evaluation of teaching.”
Other incidental factors influencing TCE rankings are age of teacher and personal appearance. Research done at Notre Dame has shown that instructors with no more than four to eight years teaching experience receive the highest evaluations, and that ratings received thereafter tend to decline. Another study at the University of Texas concluded that “attractive professors consistently outscore their less comely colleagues by a significant margin on student evaluations of teaching.” It seems that students tend to enjoy their classes more when the teacher is good-looking.
The upshot is that Notre Dame’s TCEs in their present embodiment are poor indicators at best of teaching performance. When students come to the key question (#17) asking for overall evaluation of teaching they check a box corresponding to their gut feelings at the moment. The unfortunate fact is that individual students have no better idea of how to evaluate effective teaching than did the people responsible for the design of the present instrument.
We need to realize that these shortcomings of our current TCE system cannot be remedied by changing items on the questionnaire. It is a long-standing truism that quality of education cannot be gauged by quantitative measures. Quality of life cannot be measured by income level, quality of marriage cannot be assessed by number of offspring, and quality of teaching cannot be gauged by checking boxes on questionnaires. But if standardized procedures like this will not do the job, what other means might be available?
One possibility is to put more emphasis on qualitative assessments like those students currently are asked to write in conjunction with the standard questionnaires. These qualitative evaluations take longer to compose and longer to read, in addition to being unsuited for computer processing. But they often contain information that could be useful to teachers and administrators alike.
Another possibility is to invite students to make oral reports to a neutral third party, who would convey summaries to the instructor and other persons concerned. Reports of this sort could be solicited from students currently enrolled in a given course, and then augmented by written reports from former students before or after graduation. Yet another possibility might be to enlist seasoned teachers to visit classrooms on a regular basis, and to formulate qualitative assessments to be shared with relevant parties. While such procedures inevitably would be time-consuming, their results nonetheless would probably make them worthwhile.
As long as instruments relying on questionnaires remain in place, however, administrators responsible for teaching evaluation should take steps to avoid the counterproductive effects of the present system. One step to consider is the adjustment of TCE scores according to the grading practices of individual instructors. A given score for an instructor who gives all As and A-s should not count as equal to the same score for a more discriminating grader. This expedient not only would make evaluation of teachers more equitable, but would also tend to discourage grade inflation.
Another salutary step might be to stop rank-ordering TCE scores on a percentile basis. Given the insensitivity of the instrument, an instructor ranking in the middle might be no less skilled as a teacher than someone ranking at the top of the scale. It seems a bit absurd to use percentages from 1 to 100 in grading teachers when teachers are confined to a few letters in grading students.
A final suggestion is that your administration should discourage the publication of TCE results for students to use in selecting courses. Until a realistic means is found to assess teaching performance, the TCEs should be deemphasized in University decision-making. Teaching with integrity is hard enough without having to cope with this kind of fickle publicity.
Once again, John, I wish you good fortune in your efforts as President of the world’s greatest Catholic university. I look forward to contributing whatever I can to the success of Notre Dame’s mission under your leadership. One small contribution might be to stimulate a University-wide discussion of the issues treated above. The possibility of this happening prompted my decision to make this letter available to the University community at large.
Kenneth Sayre is a professor of philosophy. This column is the last in a four-part series addressed to new University President Father John Jenkins. Kenneth Sayre can be contacted at firstname.lastname@example.org
The views expressed in this column are those of the author and not necessarily those of The Observer.