The posting below offers a look at the emerging field of what is termed "consultancy style action research." It is from Chapter 1, Educational development: research, evaluation and changing practice in higher education by Ranald Macdonald in Staff and Educational Development Series, Academic and Educational Development, Research, Evaluation, and Changing Practice in Higher Education by Ranald Macdonald and James Wisdom. Kogan Page Limited, 120 Pentonville Road, London, N1 9JN, UK. Distributed by Stylus Publishing Limited, 2283 Quicksilver Drive, Sterling, VA 20166, USA. http://www.styluspub.com/ ?Copyright individual contributors, 2002. The right of WWW to be identified as the author of this work has been asserted by him in accordance with the Copyright, Designs and Patents Act 1988. Reprinted with permission.
UP NEXT: The Impact of 'Dyslexia' (on student learning)
----------------------------------------- 1,034 words ------------------------------------
RESEARCH AND EVALUATION IN EDUCATIONAL DEVELOPMENT
Research in educational development
Research in educational development has a relatively short history, as distinct from specific research into teaching and learning, though the latter has often focused on compulsory education prior to students entering higher education. While other research into educational development has appeared over the years, the launch of the International Journal for Academic Development in 1996 sought to focus scholarly activity in this and closely related topics. In the journal's first editorial, Baume (1996) wrote that the journal's distinctive focus 'will thus be the processes of helping institutions, departments, course teams and individual staff to research into, reflect on, and develop policy and practice about teaching, learning and other activities in support of learning... The journal is intended to help define, develop and extend the practice of academic development in higher education worldwide'.
Much of the research is thus focused on practice and policy and providing the evidence for change in educational development, as part of the process of change or to judge the effectiveness of that change. The emphasis has largely, but not exclusively, been on qualitative research methods, largely borrowed from social science traditions. There has also been an emphasis in some areas on action research as a way of researching changing or developing practices. 'Action research... may be defined as collaborative, critical enquiry by the academics themselves (rather than expert educational researchers) into their own teaching practice, into problems of student learning and into curriculum problems. It is professional development through academic course development, group reflection, action, evaluation and improved practice' (Zuber-Skerritt, 1992). Beaty, France and Gardiner (1997), in advocating action research for use by educational developers 'because it involves an experiential learning cycle that fuses research, development and evaluation into a dynamic process', describe 'consultancy style action research - CSAR - as an appropriate variant because it is based on a triangular partnership involving 'the knowledge of the educational developer, the skills and time of a social researcher and the concerns and expertise of academic staff'.
There is an extensive and growing literature on educational research methods, as a glance along the appropriate library bookshelf will show. Some of the chapters in this book demonstrate a number of these research methods in action, but it is in the use of various methods of evaluation that many concentrate. However, it is not just the methods that differ - and in fact they may demonstrate methodologies equally as rigorous as much research - but also the intentions and outcomes expected. Scott and Usher (1999) note that 'evaluators are more concerned with assessing the effectiveness, or describing the impact, of a deliberately engineered social intervention'. By contrast 'researchers do not operate with such a close relationship between themselves and the initiators of those interventions, though they may still be dealing with the effects of policy interventions, though they may still be dealing with the effects of policy interventions, since these are an abiding feature of educational systems'. In the context of educational development, it is to evaluation that we should now turn our attention as this has been a major focus, rather than research per se.
Evaluation of educational development
While evaluation was once seen by many academics as a threat to academic autonomy, 'it has now come to be seen not only as a necessary adjunct to accountability, but also as an integral part of good professional practice' (Hounsell, 1999). So when developing a project or proposing an innovation in learning and teaching, the first question is often 'how will you evaluate it?'
The National Co-ordination Team (NCT) for the FDTL and TLTP produced a Project Briefing (1999) in which it links monitoring with evaluation. The reasons for monitoring and evaluation are given as being: formative evaluation to influence the future direction of the project; accountability through summative evaluation to satisfy stakeholders; and learning about teaching and learning practice and about project process, to inform future development projects. The main emphasis is therefore on whether the evaluation is formative/developmental or summative. The briefing also summarizes an evaluation strategy adapted for educational development by Baume and Baume (1995) from Nevo (1986):
1. Decide what is or are to be evaluated, and when.
2. Identify stakeholders in the project.
3. Identify stakeholders' questions and concerns.
4. Identify the criteria for judging answers to stakeholders' questions.
5. Devise and pilot the evaluation method and instruments.
6. Carry out the evaluation.
7. Report to the stakeholders
8. Change project practice as necessary
9. Review evaluation methods from time to time.
Evaluation is thus a dynamic process and not just something that happens at the end of a project or developmental activity. The link to monitoring enables those involved with evaluation to see it as part of the project process. As a past member of the NCT I was always conscious that project staff initially expected the summative elements of monitoring and evaluation to dominate, whereas the reality was that, on most occasions, it was the formative or developmental aspects which came to the fore - perhaps reflecting the background of the NCT members as educational developers.
There is not the space here to go into detail about evaluation methods but a useful source is the Evaluation Cookbook (Harvey, 1998), which was produced as part of the Learning Technology Dissemination Initiative, funded by the Scottish Higher Education Funding Council. However, most of the following questions in relation to evaluation of an educational development: Why? For whom? Of what? How? When? From whom? By whom?
The relationship between research and evaluation
Many educational researchers would question the use of both action research and evaluation as legitimate or suitably academic approaches to understanding educational developments. However, developing approaches to evaluation, partly in response to the demands of growing numbers of stakeholders for increased accountability for the spending of public funds, has meant that the line between research and evaluation has become somewhat blurred.
Chapters in this book will demonstrate a variety of approaches to evaluation, often linked to more covert research activities - the pressures of the Research Assessment Exercise in the UK are felt even within educational development projects - but still with the intention of assessing both the outcomes and process of those developments, both summatively and formatively.
Baume, D (1996) Editorial, International Journal for Academic Development, 1(1), pp 3-5
Baume, D and Baume, C (1995) A strategy for evaluation, in Directions in Staff Development, ed A Brew, pp 189-202, SRHE and Open University Press, Buckingham
Beaty, E, France, L and Gardiner, P (1997) Consultancy style action research: a constructive triangle, International Journal for Academic Development, 2(2), pp 83-88
Hounsell, D (1994) Educational development, in Managing the University Curriculum: Making common cause, ed J Bocok and D Watson, pp 89-102, SRHE and Open University Press, Buckingham
Nevo, D (1986) The conceptualisation of educational evaluation: an analytic review of the literature, in New Directions in Educational Evaluation, ed E House, Falmer Press, Lewes
Scott, D and Usher, R (1999) Researching Education: Data, methods and theory in educational enquiry, Cassell, London
Zuber-Skerritt, O (1992) Action Research in Higher Education: Examples and reflections, Kogan Page, London