INTRODUCTION WHAT IS EVALUATION? SOME COMMENTS ABOUT TERMINOLOGY

This handbook supports a project funded by the Australian Government Committee for University Teaching and Staff Development (CUTSD). The amended project title is "Staff Development in Evaluation of Technology-based Teaching Development Projects: An Action Inquiry Approach". Full details about, and an ongoing record of the project are at http://www.tlc.murdoch.edu.au/project/cutsd01.html.

The project is hosted by Murdoch University on behalf of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE), as a consortium of 11 universities.

The rationale of the project is to guide a group of university staff through the evaluation of a Computer-facilitated Learning (CFL¹) project by a process of action inquiry and mentoring, supported by the practical and theoretical material contained in this handbook. An overview of the process is shown in Fig. 1.1.

Figure 1.1. Schematic view of the project, showing roles and processes.

Participants will learn to evaluate student learning resulting from the use of their own CFL project, through a process in which they develop an evaluation plan, carry out the evaluation, analyse the data and disseminate the results. Specifically, participants will be expected to:

• Use this evaluation 'handbook', with support from their mentor, to develop their evaluation plan.
• Develop evaluation questions with the support of their mentor and other project stakeholders.
• Submit a costed evaluation plan in order to obtain a modest amount of funding.
• Carry out the evaluation study according to the timing requirements of their project (1st or 2nd semester, 2000), supported by their mentor.
• Submit regular reports about the progress of their project.
• Prepare for presentation and engagement at a workshop at the ASCILITE 2000 Conference.
• Collaborate with their mentor in the preparation of a paper for publication.

The set of evaluation studies chosen for this project is given in Table 1.1, together with the mentor assigned to each study, and a summary of the characteristics of the studies. The studies cover an extensive range of disciplines, years of study and study modes. The types of Information Technology (IT) used, and hence the nature of the CFL, vary widely. Some projects consist of one CD-based product, used in one week of semester; while others consist of a range of online resources used from week to week. There is considerable variation between these extremes. Different evaluation approaches will be needed, depending on the nature of the CFL and its educational context.

Table 1.1. A list of the evaluation studies forming this project.