ANALYSIS AND DESIGN DEVELOPMENT IMPLEMENTATION INSTITUTIONALISATION |
A LEARNING-CENTRED FRAMEWORK
FOR WHOLE PROJECT EVALUATION ANALYSIS AND DESIGN DEVELOPMENT IMPLEMENTATION INSTITUTIONALISATION |
The evaluation framework we have adopted (Alexander & Hedberg, 1994; Bain, 1999) is summarised in Table 2.1. It uses a distinction which is useful provided it is not drawn too strongly; namely between formative and summative evaluation. Evaluation that seeks to improve the project before it is fully implemented is commonly referred to as formative evaluation, to draw attention to the emphasis on the formation phases of the project. Evaluation that seeks to determine whether an innovation is worth retaining is often referred to as summative evaluation, to emphasise the need to make a judgment about the project's viability once it is up and running. In practice, these distinctions often blur, as we outline in later subsections, and we anticipate that most mini-projects will involve mixtures of both.
An aspect of the framework that is relatively unconventional is that it includes evaluation of the very first steps of project development (analysis and design) as well as of the more obvious phases that unfold as the project is being developed, implemented and incorporated into the fabric of the institution.
Table 2.1. A learning-centred framework for whole project evaluation (adapted from Alexander & Hedberg, (1994); Bain, (1999)
|
Phase |
Focus |
Purpose |
Relevance to ASCILITE evaluation project |
|
Analysis and Design |
Curriculum analysis |
To describe the inadequacies/ insufficiencies of the current curriculum, with particular attention to the shortfall in student learning. |
Necessary background material to determine the nature of the remaining evaluation and the forms of the evidence. |
|
Teaching-for-learning analysis |
To describe and justify the teaching/ learning/assessment process likely to bring about the desired learning outcome. |
Necessary background material to determine the nature of the remaining evaluation and the forms of the evidence. |
|
|
Specification of innovation |
To describe and justify the proposed implementation, and indicate how it will facilitate the desired learning process and outcome. |
Necessary background material to determine the nature of the remaining evaluation and the forms of the evidence. |
|
|
Development |
Formative monitoring of learning environment |
To determine whether the innovation is functional in its context and accessible/attractive to students (and modify as needed). |
It has been assumed that CFL projects to be evaluated have already passed this stage, and that the CFL software is mature and usable. |
|
Formative monitoring of learning process |
To determine whether the innovation is influencing the learning process as intended (and modify as needed). |
Very relevant for projects in which the CFL is used regularly by students. |
|
|
Implementation |
Summative evaluation of learning process |
To determine whether the innovation is influencing the learning process as intended. |
Very relevant. |
|
Summative evaluation of learning outcome |
To determine whether the learning outcome is as intended. |
Very relevant. |
|
|
Summative evaluation of innovation appropriateness |
To determine whether the innovation is educationally appropriate in its immediate context. |
Very relevant. |
|
|
Institutionalisation |
Impact evaluation |
To determine the robustness of the learning and its transfer beyond the immediate context of the innovation. |
May be relevant in some cases, depending on project. |
|
Maintenance evaluation |
To determine the sustainability of the innovation in the context of the whole course. |
May be relevant in some cases, depending on project. |
These two quirks aside, an important reason for using the framework, given the focus of this project, is that it places learning at the centre of the evaluation enterprise by seeking evidence that:
Each phase of the framework is discussed in detail.
