About Us

UA Home

OVERVIEW

Bob Smallwood
Assistant to the Provost for Assessment

Results from the National Survey of Student Engagement (NSSE) have been useful in helping universities identify the quality of the undergraduate learning experience on their campus.  The NSSE results fall into five key clusters of activities that research studies suggest are associated with heightened levels student learning. These Benchmarks of Effective Educational Practice include:

Level of Academic Challenge
Active and Collaborative Learning
Student-Faculty Interaction
Enriching Educational Experiences
Supportive Campus Environment

Institutions participating in the national administration of the NSSE receive a detail analysis of the survey results including customized reports of how first-year students and seniors respond to all survey items, statistical comparisons against peer group and national norms, and a comparison of institutional findings to the national benchmarks of effective educational practice.

It is not uncommon for participating institutions to subsequently disaggregate institutional results to a smaller unit of analysis, usually to the college level.  This smaller unit of analysis permits the university to localize the variation in levels of student engagement. In fact, this variation in levels of student engagement within the university is often much greater than the variation that exists between universities that make up the institution’s Carnegie equivalents.  This finding suggests that improvement initiatives might best be designed and implemented at the college level (rather than based on institutional findings) to maximally impact overall student engagement on campus.

Even when institutional results are disaggregated to the college level, it is often the case that academic department chairs and faculty within the colleges will suggest that less desirable engagement findings are the result of shortcomings in another academic department within their college rather than their own department.  “These less-engaged students are not my students” is a proclamation often advanced. And not surprisingly, it is assumed by the department chair or department faculty that high levels of student engagement within the college should be attributed to the excellence in effective educational practices that exist within their academic department. 

To validate such conclusions and/or to explore variation in levels of student engagement within the college, it would be desirable to further disaggregate the college-level NSSE findings to the department level.  However, doing so invariably results in department sample sizes that are so small that meaningful conclusions are compromised.  In response, many institutions have increased the size of their institutional NSSE sample or have targeted their over-sampling efforts to specific academic majors.

Perhaps it comes as no surprise that disaggregating NSSE results to the department level often leads to the same series of observations that occurred at the college level. That is, if less than optimal levels of engagement are observed, a faculty instructor within the department will often explain that “these less-than-optimal levels of engagement are not characteristic of students in my classes, but rather must be in other course offerings within the department. These are not my students!”

The strategy of disaggregating the NSSE results to the course level has not been an option available until now.  In completing the NSSE, students are asked to report how frequently they engage in various educational practices and activities in all of their classes during the current school year rather than a specific class conducted in a particular term. 

In order to capture relevant student engagement results at the course level, Dr. Judy Ouimet and I approached Dr. George Kuh, then NSSE Director, and received permission to develop and pilot test a classroom level adaptation of the NSSE. This classroom adaptation came to be called the CLASSE (Classroom Survey of Student Engagement).

Construction of the CLASSE

In addition to asking students in a particular class how frequently they engaged in various educational practices and activities in that class, it was also desirable to ask the instructor of the course how important he/she believed each of the educational practices are for the student to be successful in the class.  This additional focus in the CLASSE was prompted by two previous findings:

When NSSE results were shared at the college and department levels, faculty often reported they did not necessarily encourage some of the engagement practices assessed on the NSSE.  For example, in some of their classes, the instructor included assignments and exercises that called for students to work collaboratively with their classmates, but that was the exception rather than the rule.  Similarly, there might be an upper level class the instructor taught that required a class presentation, but that too was an exception rather than a typical expectation.  So from the perspective of the faculty instructor, some of the student engagement practices and activities were less important than others. It was not surprising to them then that students would report that such activities were not occurring very frequently.

This same observation and conclusion was reached by a number of institutions that used the Faculty Survey of Student Engagement (FSSE) as a companion assessment tool along with the administration of the NSSE.  Numerous items on the FSSE also ask faculty how important various engagement practices are to student success. These faculty ratings of importance can be combined with student reports of the frequency of occurrence of the various educational practices to serve as a potential diagnostic tool to set priorities for improvements within the department or college.  Instead of trying to work on many different educational practices to encourage student engagement to facilitate student learning, why not target our efforts, said some chairs or deans, to working on improving those educational practices we as a department (college) particularly value and find important, that our students report are not occurring very often.  Because such a “prescription for improvement” makes so much sense to faculty and academic administrators, this faculty development philosophy has been a central focus in the development of the CLASSE.

What is CLASSE?

CLASSE is a pair of survey instruments that enable one to compare what engagement practices faculty particularly value and perceive important in a designated class with how frequently students report these practices occurring in that class.

CLASSEStudent is the survey instrument completed by each student enrolled in the designated class, while CLASSEFaculty is the survey instrument completed by the faculty instructor of the designated class.

Rationale: Why Measure Student Engagement at the Class Level?

By measuring variation in student engagement at the classroom level, one will be able to more readily identify where that variation exists within the department, college and institution. This knowledge will enable the leadership to more efficiently and effectively target improvement initiatives to address less than optimal engagement results. It will also enable the leadership to support and reward those individual faculty, department or college efforts responsible for the more desirable levels of student engagement that are observed.

An even more potentially beneficial result from the measurement of student engagement at the classroom level is the faculty development ramifications of these measures. Preliminary results from the pilot administrations of the CLASSE have raised as many questions as they have answered, but the following findings seem to be common:

a. There is a positive correlation between the importance faculty place on a given engagement practice and the frequency students report engaging in that practice.  For example, in a given class, if faculty think it is important for students to collaborate on the completion of assignments outside of class, there tends to be above average levels of collaborative activity occurring by the students enrolled in that class. If faculty think it is not so important for students to put together ideas or concepts from different courses when completing assignments, the frequency of occurrence of such activity by enrolled students tends to be below average.

b. In every course in which the CLASSE has been administered, there are a small number of engagement practices that the instructor believes to be important that students are not doing, or doing so at below average frequency levels. Sometimes faculty have readily interpreted the causal factors responsible for these “disconnects,” but more often than not, the disconnects have prompted faculty to think about and ponder what he/she is doing or not doing in the class that might be responsible. In addition to such introspections, the instructors often begin to talk about related best practices with their colleagues, and even with campus experts that are knowledgeable about effective instructional strategies. The CLASSE results have prompted such dialogue.

c. Not only do CLASSE results vary across faculty but also within faculty. That is, what educational practices faculty perceive to be especially important in one of their classes may be quite different than what they perceive to be important in another class they teach. This result is absolutely consistent with the verbal reports we received when we talked with faculty about college and department student engagement results. Faculty would frequently remark...”how much collaborative activity I encourage depends on the course,” or ”I have students give oral presentations in my upper level class but not my lower level class,” or “I think class discussion is very important in my upper level capstone course, but only somewhat important in my lower level introductory course.”

d. Results from the administration of the CLASSE have been received very positively by the participating faculty, so much so that the faculty have wanted to repeat the assessments in all of their classes in future semesters…so much so that they’ve encouraged their colleagues to consider participating in our pilot efforts. This atypical reaction to the assessment and evaluation of instructional performance may not be as unpredictable as one might initially suspect for the following reasons:

1. Many faculty have reservations about the usefulness of typical end-of-course student feedback measures. They often call-to-question the appropriateness of comparing their results with their colleagues. Too often, they argue, the nature of the teaching learning experience is so different from one course to another or from one instructor to another that comparative assessments are empirically unsound. With this backdrop in place, the CLASSE results, and the way in which they are derived, represent a desirable supplement, if not alternative, to the more traditional approach to assessing instructional performance. The “opportunities for improvement” are not derived by contrasting results between and among faculty. What surfaces from CLASSE results as opportunities for improvement are those educational practices that faculty have indicated are particularly important in a designated class that students report not doing very often. The inappropriateness of between-faculty comparisons is further accentuated by the fact that no two CLASSEs are necessarily the same. The faculty participant has the opportunity to add up to eight unique survey items that draw attention to practices or activities he/she believes are importantly related to success in the designated class.

2. CLASSE results, particularly when represented using the quadrant analysis approach to presenting results,shift the focus of the assessment effort to improvement rather than accountability, and that focus is not only less threatening but much more constructive in facilitating student learning. “Working on strategies to get students to do more of what I think is important to be successful in my class sounds like a worthwhile objective” said one of our faculty participants.

e. Veteran users of the NSSE often devote considerable energy to deriving the “next steps” to utilize their NSSE results to prompt institutional improvement initiatives. The CLASSE affords opportunities to back-up the institution’s student engagement investment. CLASSE will help localize variation in student engagement. CLASSE,as a faculty diagnostic tool, actively engages the faculty in the student learning and quality enhancement quest.

For additional information on the background of CLASSE, see:

Ouimet, J.A. & R.A. Smallwood CLASSE– the Class-level Survey of Student Engagement Assessment Update V17, No6, Nov-Dec, 2005.

*Items #1 - #28 adapted with permission from the National Survey of Student Engagement, Copyright 2001-08 The Trustees of Indiana University