Evaluation for development: the E.A.S.Y. model for empowering actors and stakeholders in curriculum development

1 Introduction

The word judgment is very likely what comes to mind when many teachers feel their work is being assessed and evaluated. In this field, to contrast the idea of judgment, the concept of evaluation for development has been often defined in the literature as opposed to evaluation of development (e.g. Fetterman 2005). Evaluation for development, in this article, refers to evaluation that takes place in interaction and through collaboration, with a strong future-oriented focus, instead of a judgment approach or rumination on possible weaknesses and failures within the action being evaluated. In evaluation for development, both the evaluators and those being evaluated, as well as possible stakeholder representatives, interact on an equal basis with each other and together analyse and reflect on the strengths and weaknesses of what is being developed (Clavert et al. 2018; Fetterman 2005; Niemi 2013; Patton 2018; Seema et al. 2017; Välijärvi and Kupari 2010).

In Finland, all Higher Education institutions have audited quality control systems that for example detail the current feedback collection methods (Korkeakoulujen auditointikäsikirja [Auditing Handbook for Higher Education] 2019–2024). Typically, student feedback and peer feedback from fellow teachers hold a central role in developing teaching. Nevertheless, in many instances, student feedback may function as the only method for the evaluation of teaching quality. On the other hand, various peer assessment and mentoring approaches have been on the increase. For the most part, these peer assessment methods include observing a fellow teacher’s classes, a fellow teacher observing one’s own classes, and feedback discussions based on these observations. In addition, reflective self-evaluation of one’s own teaching may be used alongside the peer assessment methods (Heikkinen 2012; Lappalainen 2017).

Evaluation data can also be produced through benchmarking as well as external audits reviewing the teaching, administrative processes, and curriculum development in place (Kuivila and Kääriäinen 2016). Moreover, students’ role and student agency in the community of Higher Education has become more and more meaningful. Student-centred approaches do not only apply to teaching methods during courses, but also to developing the field of Higher Education as a whole, including the evaluation processes within it. For one, this is shown in the fact that students are nowadays more often involved in the auditing processes as full members of the auditing teams (Heiskari and Sippola 2018). This paper will describe a process of evaluation for development in a collaborative model that the authors of this paper have called kehitysriihi in Finnish, and would like to propose the acronym E.A.S.Y. for it in English (Empowering Actors, Stakeholders, and You). The focus of this paper is specifically on two aspects: Firstly, the authors’ roles as the core teachers in the process and their experiences of the E.A.S.Y. model, and secondly, the experienced empowering factor of the model. That is to say, one of the main observations and experiences was that with the E.A.S.Y. model, the emphasis shifted from analysing the individual teachers and their work to the meta level of wider-reaching pedagogical aims, as well as the students’ experience of their study path. This in turn shifted the roles of the participants in the process and enabled a more empowering evaluation to be engaged in.

2 Background for the E.A.S.Y. model

The E.A.S.Y. model was developed during a collaborative project between the Language Centre and the Law School at the University of Eastern Finland (Integroimalla paremmaksi, “Better through Integrated Curriculum”).

The project entailed the Language Centre teachers of Legal English and Legal Swedish integrating the contents and topics of selected Law courses into their language and communication courses. The students who took the courses within the project were thus able to complete both a Law course and a language course at the same time. In addition to the teachers and the students in question, also other teaching staff and administrative staff of both units took part in the Integrated Curriculum project. The core actors were the authors of this report, Dr. Susanna Kohonen for Legal English, Sinikka Kettunen for Legal Swedish, and Jonna Kosonen who coordinated the project at the Law School (Kosonen et al. forthcoming).

The aim was to develop a model and a process that would steer away from the pitfalls often related to external evaluation or external audits, such as the experience of those being evaluated on having to do extra work that does not seem meaningful or useful (Seema et al. 2017). The proposed model is based on a carefully prepared, time-efficient, collaborative panel discussion that focuses on both pedagogical development for evaluation and empowerment evaluation. It involves the actors as well as representatives of the identified stakeholders (Clavert et al. 2018; Fetterman et al. 2017; Patton 2018).

2.1 The key role of the Nutshell Poster in the E.A.S.Y. discussion panel

In the pilot example, the participants of the collaborative panel discussion were two of the core actors and teachers of the project, representatives of the Law School teaching staff and students, Director of the Language Centre, and Professor of Practice of the Law School. The core actors of the project prepared the E.A.S.Y. panel discussion carefully. The preparations included the analysis of the student feedback data. The analysis, together with the core actors’ reflective self-evaluation, were summarised and presented in a poster. This poster functioned as the backdrop and foundation for the panel discussion. The poster played such a significant role both as a tool for the reflective self-evaluation and as a tool for the panel discussion that the authors of this report consider it a key element in the entire evaluation process. It was therefore also given a name of its own, the Nutshell Poster.

The Nutshell Poster briefly described the background and the purpose of the project, together with the desired aims and outcomes, as well as the timeline and the stages of the project. Furthermore, in this pilot run, the poster overviewed the pedagogical framework of the courses that were part of the project, with a couple of practical examples of the topics covered and the working methods used in the courses.

The poster also outlined the discussion and reflection that the core actors had engaged in within their own self-evaluation and student feedback analysis. The identified strengths and successes were presented together with a critical yet future-oriented view into what would next need to be focused on and further developed. The purpose of the carefully summarised and designed poster was to enable and emphasise the role of the panel discussion. It is therefore proposed that Nutshell Posters used for similar purposes should include similar sections and elements as discussed above (cf. Figure 1). These sections cover (1) background and context of the action being evaluated, (2) aims and desired outcomes, (3) description, (4) summarised results, including an overview of what worked well and what did not work well (based on outcomes and collected feedback), (5) reflection and self-evaluation on outcomes and feedback, as well as (6) future-oriented proposals for the following steps to be taken. After the time-efficient introduction, the discussion proceeded in a flexible, free-flowing manner. The main focus turned towards the meta level from the start: How to develop the Law student’s study path as a whole, including the project that was being evaluated.

Figure 1:

The structure of a Nutshell poster. Attached as a separate file, according to the instructions.

Three main themes emerged during the panel discussion. Firstly, the panel reflected on the meaning of language and communication skills in the legal profession, as well as the required linguistic and interaction skills of future lawyers. Secondly, the discussion focused on how to implement the Law courses side by side with the language and communication courses in order to enable and strengthen the communication skills required in the legal profession, specifically through the collaboration of the Law School and the Language Centre. Thirdly, future scenarios on three-partite co-operation were sketched, involving not only the Law School and the Language Centre teachers, but also contributions from legal experts and representatives of working life.

The language teachers whose integrated courses were evaluated during the panel reflected afterwards on how they experienced the panel discussion. Due to the future-oriented approach of the panel, they felt they were on an equal ground with the other participants in the panel and not mere objects under review. In other words, they felt they gained the experience of agency also in the process of evaluation, which is one of the aims of evaluation for development, or empowerment evaluation.

3 Discussion

The authors of this report propose that in the midst of other work engagements and fully booked schedules in higher education, the evaluation for the development model described here could prove to be an invaluable asset. Evaluation of education must not be left to external auditors only. In order to engage the actors themselves – teachers in this case – in the process of evaluation of their own work, the authors maintain that the meetings of the E.A.S.Y. panel discussion sessions should not exceed 1 hour. It may otherwise become very challenging to find the required stakeholder representatives to participate in the panels.

Moreover, the authors of this report argue that the careful preparation, design, and use of the Nutshell Poster was a key factor in the subsequent success of the E.A.S.Y. panel discussion. Producing such an effective poster required appropriate evaluation data and time to analyse it. The principles of evaluation for development involve collecting and analysing evaluation data throughout the duration of what is being evaluated, or by adapting and combining different methods of both producing and analysing the evaluation data (Patton 2018).

In the example project, student feedback was collected as continuous feedback and with a multi-method approach. The collection methods included non-structured small group discussions during the courses, as well as a focus group interview of three students who took the courses, and the written feedback survey used at the University of Eastern Finland, at the end of the courses (in accordance with the decision of the Rector of the University of Eastern Finland 1537/12.00.00/2010). The acquired data were analysed using qualitative content analysis, accompanied by the teachers’ self-reflection and self-evaluation.

The authors of this report suggest that the proposed collaborative, participatory, and multi-method approach, based on equality of participants as well as the principles of formative assessment, is more fruitful and constructive than an audit-type summative assessment approach that may focus only on the end results of a project. Furthermore, the proposed approach and model enable and enhance communicative evaluation and boundary-crossing collaboration. Moreover, the proposed model offers a shift from a judgmental point of view into a future-oriented, negotiation-based approach, and thus increases the actors’ professional development and well-being at work. All the above-mentioned factors in turn strengthen the sense of community, the sense of having one’s voice heard, as well as the sense of ownership and agency of the actors (Niemi 2013, 2015; Niemi et al. 2018; Seema et al. 2017).

As the focus was not on what may not have been very successful in the past, but rather on shifting the emphasis to the future aims and objectives and how they could be reached, the core actors and core teachers experienced that the evaluation process enhanced their collaborative and collegial learning by strengthening their experience of agency. On the whole, the core actors reported that having participated in the E.A.S.Y. discussion panel, first of all, increased their motivation for continuous development of their teaching; secondly, it enriched and improved collaborative, collegial and cross-boundary learning in the units that were involved in the process; and thirdly, it facilitated future-oriented, democratic, and community-based organisational learning (Clavert et al. 2018; Fetterman 2005; Niemi 2002, 2015; Niemi et al. 2018; Seema et al. 2017).

4 Conclusion

This report has described and proposed a possible model, E.A.S.Y. (Empowering Actors, Stakeholders, and You), for curriculum development in line with evaluation for development. The example project, during which the authors of this report created the model, took place in the context of Higher Education but it could easily be adapted to other contexts. The panel discussion, as part of the E.A.S.Y. model, promotes and establishes an interactive, participatory, democratic as well as future-oriented approach into curriculum development. Moreover, the authors maintain that the model could be suitable to any context in which the aim is to increase motivation for development through enabling and strengthening the agency of the participants. In particular, they experienced the E.A.S.Y. model as a useful, motivating, and empowering tool to develop their work. This experience has led them to reconsider the usefulness of more widely approved evaluation methods in the field of education, such as external audits, accreditation, benchmarking, or the usual systems of written student feedback collection at the end of each course. They suggest that, for more impactful outcomes, these widely approved evaluation methods should be combined more often with multi-method feedback collection. Furthermore, based on their experience, they suggest that in order to put into practice effectively what is raised in the student feedback, or in the external audits, more empowering and participatory methods, such as the proposed E.A.S.Y. model with its Nutshell Poster, should be employed alongside feedback collecting and external auditing.