Evaluating Open and Distance Learning, Mary Thorpe, New York: Nichols Publishing, 1988, 210 pages. |
This is not a book for easy chair reading, but it may well belong on the shelf of distance education instructional designers or distance education program managers. Ms. Thorpe begins with irrefutable arguments regarding the need to include evaluation at all stages of the development and delivery processes in distance education. Practitioner evaluation need not replace outside "objective" studies and analysis, but evaluation in the development process and throughout delivery activities allows for the immediate implementation of large and small changes to make distance education materials and programs more effective. There is undisputed value in instituting evaluation of course materials at the formative and summary stages by instructors, designers, tutors, and students (or potential students), but the author also suggests that evaluation of the learning environment include input from office staff in daily touch with learners, technical, and support staff from learning centres and from former students, administrators, counsellors, and so on.
Ms. Thorpe indicates that evaluation is apparently only taking place at the Open University and that she could find no other sources for studies of this type. She evidently has not looked beyond the borders of the U.K. to other jurisdictions where some excellent studies have been undertaken. This rather provincial search for evaluation studies is a minor criticism of the book, however, because the chapter giving examples of some of the evaluation projects from the O.U. is perhaps the least interesting and useful in the book.
The evaluation recommended by the author is that which practitioners can perform from their present knowledge and skills. She suggests that undertaking what intelligent evaluation one can do is more practical and useful than learning any particular new research approach or making use of a specific theoretical model or paradigm.
If practitioners need ideas on what evaluation they might institute, a quick look at the detailed Table of Contents will suggest studies that can be under-taken at various stages in the development and operation of a distance education program. For example, learner self-evaluation may be undertaken before beginning a program to clarify the student's own motivation and suitability to undertake a program or during courses to clarify the student's learning style, competencies, and motivation. Another chapter indicates many methods and approaches for the practical evaluation of all stages of the tuition process including grading turnaround time and reliability as well as effective tutor correspondence and grading comments.
Practitioners involved in the counselling area are given suggestions using routine records, interviews, and questionnaires with which evaluation may be conducted. Samples of forms, tables, appropriate questions, and results are numerous. These would be especially helpful as a practitioner begins to design an evaluation instrument, in order to "get started" with some tested, well worded, general questions and areas to study.
In the chapter on evaluating course and learning materials, both quantitative and qualitative approaches are explored. The experience of tutors with the materials is stressed as important both for the improvement of specific courses (where student difficulties have been found and need clarification or revision) and for generalizing to the development of new course materials. Learner feedback can serve to locate difficult areas, while peer (other expert) feedback can suggest alternative approaches because "on its own, learner feedback can leave us knowing there is a lot wrong with a course, but without a coherent idea of how to put it right." A "pro forma" for the assessment of the quality of existing open learning materials to be completed by students would be useful in identifying the appropriateness and effectiveness of a course; this form could be adapted for most distance courses.
The last chapter of the book indicates methods of analyzing what the evaluation has demonstrated and charts potential uses for the results of evaluation. The conclusion indicates one caveat regarding the evaluation process: it must be used to improve the learning environment and to control quality of the learning environment, but control must not be so prescriptive and inhibiting that improvements cannot be easily instituted in course materials and in the operation of distance or open learning programs.