Tomorrow's Teaching and Learning
The posting below looks at a new tool for assessing classroom engagement. It is by James Rhem, executive editor of the National Teaching and learning Forum and is #38 in a series of selected excerpts from the NT&LF newsletter reproduced here as part of our "Shared Mission Partnership." NT&LF has a wealth of information on all aspects of teaching and learning. If you are not already a subscriber, you can check it out at [http://www.ntlf.com/] The on-line edition of the Forum--like the printed version - offers subscribers insight from colleagues eager to share new ways of helping students reach the highest levels of learning. National Teaching and Learning Forum Newsletter, Volume 16, Number 4, May 2007.? Copyright 1996-2007. Published by James Rhem & Associates, Inc. All rights reserved worldwide. Reprinted with permission.
UP NEXT: Stronger Presidents for Tough Times?
Tomorrow's Teaching and Learning
---------------------------------------- 1,404 words ---------------------------------------------
CLASSE - The Missing Link?
James Rhem, Executive Editor
Understanding of teaching and learning has increased significantly in the last several decades, but it's done so in pockets. The Seven Principles occupy one, Active Learning (and the many pedagogies within it) another, and evaluation and assessment in still others. The evolution of useful understanding remains in search of a missing link to tie them all together.
Could CLASSE constitute that link? Perhaps, though its authors are too modest to suggest it.
CLASSE is a little-known cousin of NSSE (the National Survey of Student Engagement) based at Indiana University. NSSE began in pilot studies at 70 campuses in 1999. Most recently 610 campuses participated. NSSE empirically assesses student participation in programs and activities - confirmed "good practices" in undergraduate education - that institutions provide for their learning and personal development. The aim, of course, as the NSSE website says has been "to identify aspects of the undergraduate experience inside and outside the classroom that can be improved through changes in policies and practices more consistent with good practices in undergraduate education."
Though presentations of the data can be mind-numbing, NSSE and the concept of "student engagement" have been well received. However, like many probing studies conducted on such a large scale, the results have stirred up the world of administrator's conversations more than they influenced faculty practice in the classroom.
Robert Smallwood, associate provost at the University of North Florida and one of the authors of CLASSE along with former associate director of NSSE Judy Ouimet, reports that when unfavorable NSSE data on student engagement came back, faculty often responded "These less-engaged students are not my students." That reaction at every sublevel or disaggregation of the survey data: if the university got low marks, it was the school of X that had the disengaged students; if it was the English department, it had to be 19th century area that had them. That was the usual reacting the first time a campus administered the NSSE survey. "Usually the second time around campuses administer the FSSE after doing the NSSEE when the results disturbed and bewildered them," says Smallwood.
"It happens with maturity," says Smallwood. "The real thrust with the NSSE came in 2002. There was a process of denial [greeting its reception]; so there was an effort to ask faculty the same questions and hope we get the same result or be able to convince them there is something going on that needs attention. It leads to very healthy conversations about teaching on the whole."
A Faculty Voice
The FSSE or Faculty Survey of Student Engagement, say its authors, is "designed to measure faculty expectations of student engagement in educational practices that are empirically linked with high levels of learning and development."
Introducing FSSE did two things for the conversation. First, it opened an evidentiary dialogue between what students said they were doing and what faculty thought they were doing or encouraging. Second, to some extent it opened a dialogue between faculty's own sense of "good practices" and the subtext of "good practices" informing the design of NSSE.
"First item on NSSE is how often do you ask questions in class?" says Smallwood. Faculty would say, 'Are you suggesting that the more students ask questions in class the more they are learning or are engaged?' Well, it's not necessarily true. It's not the sheer frequency of any of these behaviors. The more of it doesn't necessarily mean higher levels of student learning," Smallwood continues. "There's a general positive relationship between these behaviors and student learning, and there is a strong correlation between engagement overall and student learning. So, it's appropriate to be suspicious of these simple relationships there because they aren't simple."
Faculty were suspicious, but some were also interested. If a wide-angle lens was showing something interesting, but not in enough detail, why not use a smaller lens? Why not look at specific courses? Smallwood and Ouimet's CLASSE does exactly that. Paired with the FSSE, it gives faculty concrete data comparing their own sense of what constitute the best practices or pedagogies for evoking student engagement in a specific course compared with students's assessment of how often they are participating in them in the course. For the most part, the CLASSE follows the NSSE structure, but it offers faculty two kinds of latitude. First, because faculty have a sense of how important discussion may be in a particular course - and reflect that in their pre-test equivalent for the course, their FSSE - they can evaluate their students's responses intelligently in the context of what they are doing in class. Second, the CLASSE affords faculty the opportunity to create eight unique items specific to their course and how they are teaching it.
Clearly, when FSSE and CLASSE results differ markedly at this scale, something in the instruction needs further investigation and probably some adjustment. And at this level, faculty have embraced the findings. "It's the first time in my career," says Smallwood, "that I have faculty calling me asking me to come to evaluate their class. This has been unheard of in my 25 years. Giving the CLASSE and putting it together with the FSSE, faculty see that as a lot more valuable than the standard end of semester student evaluation."
While Smallwood has focused largely on administering CLASSE at the end of a semester, his colleague Judy Ouimet has taken a more aggressively formative route. She offers CLASSE early in the semester so that if adjustments need to be made, they can be made before the course ends. Now working with the assessment office at the University of Nevada-Reno, Ouimet has had the chance to give CLASSE to thousands of students and the FSSE to dozens of faculty in psychology and math courses. In those courses, on that campus, no wide disparities between faculty intentions and student perceptions have emerged though as might have been predicted, levels of student engagement were higher in classes where the instructional format included discussion.
>From the start, Ouimet has been keenly interested in extracting as much specific understanding from CLASSE results as possible. She's wanted, for example, to learn if there are disciplinary patterns of engagement linked to specific instructional strategies. The whole concept of "engagement" offers intriguing questions about student learning. "When there is a classroom activity like a lab or a presentation," says Smallwood, "the relationship between engagement and learning is very strong. It's still positive, but less strong on pencil and paper sorts of things like tests and papers." Back in Reno, Ouimet is currently working on linking student level CLASSE responses to learning indicators such as pre- and post-test gains, final exam scores and grades, to see if students who indicate high engagement levels also have higher learning outcomes. And, good researcher that she is, Ouimet will correlate that data with ACT/SAT scores, gpa's and so on, to help control for student ability level as well.
But the good news about CLASSE is that using it and benefiting from the findings need not wait on administrative initiative. Faculty can do it themselves. They need only contact NSSE central to obtain free copyright permission. Indeed, to get the ball rolling Ouimet suggests to faculty that at the very least they simply add key CLASSE questions to the required end of semester student evaluations. That would require less work and begin to forge a stronger link between what faculty are trying to effect and what students see happening.
For those interested in doing full-scale CLASSE, Ouimet has moved to automate the process of scoring and evaluating the data, but that information isn't yet available from NSSE central.
One of the most encouraging things one feels in talking with Smallwood and Ouimet comes from the grounded, practical good sense they exude. Ouimet, for example doesn't want to see a time when CLASSE would be an institutionalized requirement. "Students could become over-surveyed and their answers less meaningful," she says. She thinks that's already happened with class evaluations. "By the time a student is a senior, I don't think they pay any attention to them," she says. Indeed, it's a respect for dialogue more than data that fuel Ouimet and Smallwood's efforts. And Ouimet believes students must be in on it; that's part of why she favors offering CLASSE early in the semester. "Students must have some feedback on what happened," she says. "We have to close the loop for students and faculty."
Currently, while NSSE central licenses the free use of CLASSE, it is not mentioned among the related projects on its website (http://nsse.iub.edu/index.cfm).
For information and permissions, contact NSSE Associate Director Robert Gonyea (firstname.lastname@example.org) or Assistant Director Todd Chamberline (email@example.com).
For an overview of CLASSE go to http://www.unf.edu/acadaffairs/assessment/classe/overview.html
For a look at a CLASSE survey (this one for a senior literature class) go to https://websurveyor.unf.edu/wsb.dll/53/CLASSE_StudentJones.htm
University of North Florida
4567 St. Johns Bluff Road, South
Jacksonville, FL 32224-2645
Telephone: (904) 620 2784.
University of Nevada-Reno
University Assessment, Mail Stop 087
Reno, NV 89557
Telephone: (775) 682.5459