Skip to content Skip to navigation

Using Action Research to Evaluate Courses and Teaching

Tomorrow's Teaching and Learning

Message Number: 
1521

Rather, we are interested in what teachers can do to satisfy themselves about the courses they offer and their own performance.  In other words, we are interested in ‘action research’.

Folks:

The posting below looks at using a process called “action research” to evaluate your courses and teaching.  It is from Chapter 7 – Evaluating Teaching: Future Trends, in the book, Teaching & Learning English Literature by Ellie Chambers & Marshall Gregory. Published by SAGE Publications Ltd, 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP. www.sagepublishing.com ©Ellie Chambers and Marshall Gregory, 2006. All rights reserved. Reprinted with permission.

 

Regards,

 

Rick Reis

reis@stanford.edu

UP NEXT: Working in Groups and Facilitating Discussions

 

Tomorrow’s Teaching and Learning

---------- 1,889 words ----------

Using Action Research to Evaluate Courses and Teaching

Here we focus on some of the issues surrounding the evaluation of courses and teaching as it affects academics as teachers. We saw earlier in the book that in many countries regimes for assuring quality of provision at the levels of the department and institution are centrally inspired and controlled: audit or inspection systems, with requirements for data of certain kinds to be gathered by university authorities from staff and students.  For instance, universities in Australia and the UK routinely collect and analyze huge data sets of, variously, patterns of student enrolment and attendance, dropout rates, continuous assessment and exam pass rates, grade distribution patterns, employment destinations of graduates, and so on.  And routinely students are surveyed to gauge their satisfaction with their courses and their teaching as well as with university facilities such as library, study accommodation, etc.  We are not concerned here with this kind of mega-inquiry, which in any case will be carried out by educationalists, statisticians and other suitably qualified people according to whatever governmental and institutional policies prevail.  Rather, we are interested in what teachers can do to satisfy themselves about the courses they offer and their own performance.  In other words, we are interested in ‘action research’.

Action research

This type of inquiry involves teachers both investigating a course in action throughout, and reflecting on their own actions (see Schön, 1983 – and Bleakley’s critique of Schön: ‘“Reflective practice” is in danger of becoming a catch-all title for an ill-defined process’ (Bleakley, 1999: 317)).  It is formative inquiry, aimed at improving aspects of the course as it proceeds or next time round, in which teachers compare their aspirations and intentions to what actually happens on the ground.  But of course what ‘actually happens’ can’t simply be read off the surface of events. Things may not be quite as they seem, especially when the inquirer and the course designer/teacher are one and the same person.  So, various strategies and techniques have been proposed for carrying out good research of this kind more objectively. [Note 1] But it may be that in this context ‘research’ is too grand as a term for the approach teachers want to take to monitoring and improving what they do.  A number of options are open to us, from the informal all the way to full-blown investigation.

Self-monitoring

Being observant about how students are engaging with the course texts and teaching methods, what goes on in the classroom or online and the students’ progress in assignment work and so on, and mulling over what we might do better, is at the informal end of the range of possibilities.  And these are things that surely all of us do while going about our business, as intelligent people who care about the quality and effectiveness of our teaching and our students’ learning. But as an aid to such critical reflection, or in order to conduct it on a more objective and secure footing, some teachers find it helpful to go one step further: for example, to keep a journal in which they record the main events of the session (or week, month) along with their thoughts about these things and any ideas they have for future action.  We are not talking here only about recording teaching-learning problems or ‘failures’. It is equally important to record strategies and events that worked well, and to reflect on the reasons for their success, so that they can be replicated with confidence.

Beginning teachers may find this idea particularly helpful, and even more so if they can team up with a Literature colleague, or better still a group of them, to talk over some of the issues that emerge.  Often, teaching is not discussed much in the staff room, unlike one’s research, so being open about these matters no doubt takes a bit of courage to begin with.  Indeed, it may be foolhardy or downright impossible in departments where the fiction is maintained that teaching is not only ‘no problem’ but is relatively unimportant, and where any attempt to discuss it is taken to reveal the academic’s failings and unworthiness for tenure or promotion.  Sadly, this happens.  However, to us, the advantages to be gained from a group of people discussing the issues at the heart of their work – especially people with a similar, and cooperative, job to do – are blindingly obvious.  A perhaps less obvious but even more valuable outcome in the long run is that the community of Literature teachers may develop a shared ‘language’ for talk about their pedagogic practice that is both apt and enabling.

Peer review

In departments in which teaching is rightly valued, colleagues may be able to enlist each other’s assistance somewhat more formally, as reviewers of proposed course designs (syllabus, teaching-learning methods and assessment practices) who can offer critique and helpful advice before a course starts.  Once it is underway, a sympathetic colleague may then act as an occasional observer in the classroom, recording his or her observations of particular incidents and discussing them subsequently with the teacher.  This is a convincing way of having one’s own teaching strategies and habits uncovered, and a more productive way of learning about teaching in general than being offered injunctions or prescriptions about it by disinterested parties.  Examining video recordings made of teaching sessions can achieve similar results, especially when they are viewed and discussed by the teacher in the company of others.

In saying here that these are rather more formal roles for colleagues, no suggestion of surveillance is in any way intended.  Subjecting a course and teaching to scrutiny by colleagues is a way of making reflection on one’s own activities a more objective and fruitful process, and it depends upon trust. Any hint of ‘evidence’ being gathered or of management’s interest in this process would of course scupper the very possibility of openness – and of acquiring the insights into teaching that can benefit all concerned.

Student review

Quite apart from information that the department and institution will gather from students, teachers themselves may well want to check whether their perceptions of the course design, of their teaching and the students’ learning, are similar to those of the students involved (and of a colleague-observer’s, if this has been solicited).  Such comparisons between the views of different parties is known as ‘triangulation’, a process that yields a more rounded and hence reliable view of whatever is being investigated.

To gather the students’ views, teachers may ask them to complete a short questionnaire (anonymously) about their experience of the course at or near its end, whether in class, online or in their own time. The teacher may ask which parts of the syllabus the students most enjoyed studying and think they learned from well (and vice versa), which teaching methods/sessions they found most interesting and helpful, what they thought of the essay and other course assignments and so on.  Depending on the size of the group, the questions may be ‘open’ (students write in their answers) or ‘closed’ (e.g. ‘yes/no’ answers or ticking one response among several given options).  Open questions have the advantage of allowing students to raise the issues that matter most to them, to illustrate and give reasons for their views. But, as this suggests, far too much information may be generated, or information that is too diffuse for the teacher to analyze comfortably when the class is large. And because the students have to write their responses, this is a time-consuming task which they may complete only perfunctorily or not at all.  Coded questionnaires of course take less time both to complete and analyze, but although they allow the teacher to sum the students’ ‘votes’ they do not yield much information beyond that.  And the teacher receives answers only to the questions he or she has thought to ask. So a questionnaire that combines closed and open questions may be the most satisfactory option.

Another way of canvassing students’ views is to hold a discussion about the course among a representative group, a ‘focus group’, either face to face or in computer conference.  In such discussion it is obviously easier to follow up an observation or criticism and really to get to the bottom of any perceived difficulties.  And groups will, in the nature of things, tend to reach a consensus – or, at least, any major differences of opinion will be clear enough.  But the main drawback is that the students are identifiable so they may be reluctant to be honest about their experiences and judgements.  A way round this is to have someone other than the teacher conduct the discussion, but this person will not be as familiar with the course as the teacher and may not pursue the issues that would be of most interest to the teacher.  In any event, if the discussion is in person it must either be tape recorded or notes must be scribbled down as fast and as comprehensively as possible while it proceeds; a computer conference has the advantage of providing a transcript. And analyzing the outcomes will be time-consuming.

Conflicting messages

But what if on some issues no consensus emerges from the students’ responses in questionnaires or discussion, or they even conflict? What if, on reflection, teacher A does not agree with teacher B’s judgement about some aspect of her teaching they are reviewing together on video tape? How does the teacher decide which views to accept, which to act upon? The notion that a teacher may not agree with another’s perceptions of an event gives us a clue here, for that disagreement implies certain criteria of judgement.  That is, teacher A has reasons for her disagreement with teacher B, and these reasons are most likely to be based on her broad (even if unarticulated) philosophy of education and the teaching aims that flow from it.  To take an obvious example, if teacher A believes that teaching should be learner-centered she is unlikely to be persuaded by teacher B that it is a waste of everyone’s time asking the students to discuss together a matter about which they know very little.  Quite rightly, what teachers do in this situation is accept feedback and advice that accords with the educational beliefs that guide their actions, or at least does not conflict with them.

The corollary of this is that it behooves practicing teachers to be aware of and to be able to articulate their underlying beliefs. What evaluation then reveals is any differences between teachers’ ‘espoused theories’ (what they think they believe) and their ‘theories-in-action’ – what they actually seem to be doing in practice (Argyris and Schön, 1974).  In this connection, a (suitably adapted) ‘cycle’ of monitoring and reflection such as that proposed by Kolb (1984) may recommend itself: do some teaching; reflect on the experience / gather others’ views about it; conceptualize it, including reading the relevant educational literature; plan future strategies … and so back again to the beginning of the cycle.  But perhaps this sounds like a lot of effort – yet more planned, purposeful activity on the teacher’s part that begins before a course starts, continues throughout it and, seemingly, never ends – when already the academic’s job (teaching, research, administration and service to the wider university and society) is overwhelming.  So, teachers might be inclined to ask, why go to all that trouble?

Bibliography

Argyris, C. and Schön, D.A. (1974) Theory in Practice: Increasing Professional Effectiveness. San Francisco: Jossey-Bass.

Bleakley, A. (1999) ‘From reflective practice to holistic reflexivity’, Studies in Higher Education, 24(3): 315-30.

Kolb, D. (1984) Experiential Learning. New York: Prentice Hall

Schön, D.A. (1983) The Reflective Practitioner. New York: Basic Books.