By Linda Hartley, School of Education and Health Sciences; Jonathan Hess, College of Arts and Sciences; Kurt Jackson, Physical Therapy

Because the majority of faculty have been hired since our current Student Evaluation of Teaching (SET) instrument was created and implemented, this article was written to provide background information on its development. It may also serve as a reminder for the purpose and usage of SET. 

Prior to 2014, the student course evaluation instrument used campus-wide was identified as outdated, not informed by best practice, inappropriate for some classes and teaching approaches, not serving both summative and formative purposes well, too long, reliant on paper/pencil vs. online, and not suited for appropriate data analysis. After two years of discussion in the academic senate, the senate approved a recommendation to revise the student course evaluation instrument (DOC 2012-03). The SET committee was formed and regularly met for 1.5 years.

The SET committee procedures included an extensive review of literature (research, best practice, other institutions’ processes and SET instruments); multiple reports and updates to the provost and academic senate with discussion and feedback; instrument item testing and analyses; three semesters of pilot testing; collecting feedback from chairs, students, faculty; forums for SET discussion and feedback (chairs’ collaborative, Faculty Exchange Series, faculty open forum); consulting with two nationally recognized SET experts and their extensive reports; and exploring in-house and external vendors for online SET delivery. 

When developing the specific items used in the new SET, we considered a number of factors. First, we selected items informed by the literature that have consistently shown to be associated with teaching effectiveness and learning, which includes 1) teacher preparation and organization; 2) the learning environment; 3) the student-teacher relationship; and 4) learner outcomes. We initially came up with 19 items to assess these four key areas with the hope that this could be cut to six-10 core SET items following pilot testing.

Pilot testing was conducted during three different semesters with diverse courses and programs from across the university. A variety of factor analysis methods were then used to minimize redundancy of the items which led to the current eight core items. Once these eight items were selected, we used both initial and later pilot tests to check items for bias with certain known variables such as: class size, percentage of students completing evaluations, course level, date evaluation was submitted, students’ class year, students’ interest in the course, students’ current GPA, students’ anticipated grade in the class, sex of the student and instructor, and the course’s academic unit. None of these potential biases was deemed a significant influence following the analysis.  

In conjunction with the development of the SET, online resources were also created that align with the key elements of effective teaching that can be used by faculty for formative purposes. There is also online guidance and recommendations for faculty and chairs of how SET data should be used most appropriately for summative purposes as one of the multiple measures of teaching.  

The final report and recommendations were made to the academic senate in September 2013, with the proposal for the new instrument (DOC 2014-02) receiving senate approval. The first campus-wide use of the new instrument began in Fall 2014.

SET Fun Facts

107,186 surveys were issued to UD students during the 2018-19 academic year.

• The same SET survey instrument is used for all academic programs.
• Response rates are highest when surveys are conducted in the classroom.
• For one semester students submitted the equivalent amount of written responses to 18 copies of War and Peace.
• UD’s implementation of SET has been used as a role model by other US universities.