EFQM model implementation in a Portuguese Higher Education Institution

2.1 Quality management on Higher Education Institutions (HEI)

Despite the raised concern, regarding HEI quality around the world, and over time, it is in the last few decades, that this is more [5]. In its genesis, there are essentially factors such as growth and exponential appearance of HEI all over the world, and changes in the scope of their supervision, whether private or public, with governments having a supervisory role, rather than control, resulting thus in an increase in the autonomy of these institutions [8].

However, Quality as a concept (and in HEI in particular) is not easy to define. In fact, the various debates about the quality of higher education, have revealed some difficulties in obtaining some consensus, not only on the definition of quality itself, but mainly on its implications for the higher education [9]. According to [10], one of the main factors for the lack of consensus in its definition, is the multidimensionality of HEI.

This feature is reflected in the wide variety of missions, normally associated with various stakeholders, with HEI, which allowed the creation of even greater dynamics than a few years ago. The dynamics created, although positive in many aspects, contributed in part to a, wear of trust in the higher education system [5]. This "wear" has been studied in most countries and it has promoted the discussion around the concept, as well as the quality assurance activities in institutions and higher education systems [11].

In this context, countries such as Netherlands, Flanders and Portugal, whose HEI were respectively in charge of the national evaluation system, have seen their governments transfer this function to a set of an independent accreditation agencies (respectively for each country) for recognizing that they would provide the necessary results, free of any interest or internal pressure. In Portugal, this agencie is called Agencia de Avaliação e Acreditação do Ensino Superior (A3ES).

According to A3ES, Quality (in higher education) can be defined as follows: "Multidimensional concept, multilevel and dynamic, related to the context of an educational model, with the institutional mission and objectives, as well as with the rules and the specific terms of reference of a system, institution, course, program or disciplinary unit [12].

According to [13], quality can take on different meanings (sometimes conflicting with each other), and which depend essentially on:

1. Perspective of different stakeholders in higher education (e.g. students, teachers,

2. disciplinary areas, labor market, society, government);

3. Internal references (inputs, processes, outputs, missions, objectives, etc.);

4. Attributes or characteristics of the academic world to be evaluated;

5. Historical period in the development of higher education;

In addition, the promotion of Quality in HEI, through the evaluation of their performance, implies the creation of organizational structures, models and indicators, which support a culture and dynamics of their own, and although not rooted in HEI, are essential in promoting of evaluation cycles, helping institutions to take responsibility to their stakeholders [14].

It is evident, then, that quality is one of the most relevant aspects of higher education reform around the world at a time when the reduction of public funding to HEI is increasingly evident [15].

In this sense, the European Union, through the European Quality Area in Higher Education, has established directives for each member government to adopt measures for the promotion and accreditation of quality in higher education, a challenge to which Portugal responded with the creation of the A3ES [5].

In this context, the importance of accreditation in HEI can be evidenced through the definition of the USA (European University Association) group, whereby accreditation is a formal government published regarding the quality of an institution or program, following a cyclical assessment based on agreed standards [7].

In addition to accreditation, it is important in the first instance to emphasize the role of evaluation in HEI, so HEI, in a culture of anticipation, cannot afford to do without this purpose.

According to [15], the evaluation of higher education can be defined, such as: "Systematic and critical analysis process for the issuance of judgments and recommendations on the quality of a higher education institution or a cycle of studies ". The key concepts implicit in this definition were obtained by the Agency, based on the work of [16], and later adjusted to the HEI’s Portuguese context.

Several studies have been developed in this field (e.g. [17]).

The evaluation generates learning, promotes professional and personal change, so it takes a prominent place in policy discussions and in the management of HEI itself.

The question of evaluation is directly related to the decision making, carried out within the organizations, establishing a place for critical dialogues, allowing in this way to perceive the multiple references that sustain the way of acting of the various involved parties.

The evaluation, being internal and / or external, serves the organization in the sense in which the evaluated ones are also evaluators.

The internal and external evaluations are complemented when the institution, after being submitted to an internal evaluation, will have the respective results explained in a report that will serve as a point of support for the external evaluation [18].

It is in this context that the A3ES assumes special relevance, contributing to the improvement of the quality of higher education in Portugal through the evaluation and accreditation of HEI and its study cycles, within the scope of its mission.

Still in relation to the internal evaluation, the process of self-evaluation will only succeed after all the participants understand and share the same theoretical framework.

On the other hand, [19] reinforces the importance of internal evaluation by recommending that organizations should seek to implement IQAS to improve quality, regardless of the external quality assurance systems to which they may be subject.

In recent years, model proposals have emerged that meet these requirements and seek to reduce differences and develop some consensus around the practice of quality assessment [9]. Some of these examples are the Benchmarking Exercises [20], the US Institutional Evaluation Program [21] and the EFQM - Excellence Model [8].

The first two propose a holistic evaluation of HEI considering not only the teaching and scientific research developed, but also the management of institutions, while the EFQM model, based on Total Quality Management (TQM) and with evidence given in the industry, begins to be introduced, and essentially, in the management of HEI, a little around the world [22].

2.2 Internal Quality Assurance Systems (IQAS) on HEI: The Portuguese context

In Portugal, the entity responsible for promoting and evaluating quality in HEI, is the Agency for Assessment and Accreditation of Higher Education (A3ES). This agency comes from the initiative of the Portuguese government, when it approved in 2007 the new legal regime for the quality of higher education (Portuguese law Decreto de lei nº 369/2007, of 5th November), based on European recommendations, published by the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)^[3]. This agency has the objective of evaluating and accreditation of HEI and their study cycles.

This legislation sought to give the HEI some autonomy regarding the responsibility for quality assurance in their institutions (in part ensured by the freedom to choose the adopted IQAS), while ensuring that the Government retains the power to demand HEI responsibility quality. The certification of the IQAS of the Portuguese HEI is also a competence of the A3ES, and ultimately the agency has developed a strategy to make the quality assessment and accreditation system of study cycles more flexible. The main objective is to enable HEI in general and through their audited and certified IQAS to be addressed in a lighter manner, supported by institutional audits and accreditation of only a sample of study cycles [23]. For the implementation of the IQAS, the A3ES developed a set of ten references, which act as recommendations for the implementation of the IQAS in the HEI in Portugal. The use of these references by the HEI, and subsequently the accreditation of their IQAS, constitutes a powerful instrument of consumer protection, and at the same time, helps to consolidate the principles of Bologna Process [3].

In the audit process carried out by the A3ES, it is assumed as a fundamental assumption, respect for the HEI’s autonomy, and the main objective of this process is essentially the strategic institutional evaluation for the quality and the way it translates into an effective IQAS and well documented.

There are currently 19 HEI with IQAS, accredited by the A3ES. On ANNEX I, there is a list with the Portuguese HEI currently accredited by A3ES, as well as the number of years of accreditation and the date of publication.

Analyzing data from ANNEX I, there are currently six types of HEI certified, namely: 1) two nursing schools, 2) six polytechnics, 3) six universities, 4) an organic unit (OU) of a university , 5) a private institution and 6) an Instituto Superior de Estudos Militares (the Portuguese Military Academy). Fourteen of them, obtained accreditation for a period of six years, which means that approximately seventy percent received the highest certification from the A3ES.

On each HEI certified, there is a Quality Manual, that aims to describe the HEI IQAS, defining its organization, behavior, actors and respective responsibilities and present the institution’s Quality Policy.

It should also be noted that in both of the aforementioned manuals and associated with the IQAS, the roles and responsibilities associated with the different actors in the IQAS are described, as well as the participation of the different stakeholders of an HEI, namely: teachers, students, collaborators, alumni, companies and other external entities, and it is fundamental for each of these stakeholders to monitor their degree of satisfaction.

2.3 Advantages and difficulties found on literature, regarding IQAS’s application

In addition to the diversity of stakeholders with different perceptions and requirements in the scope of quality, other difficulties of implementing the IQAS can be pointed out, namely the fact that the organization’s employees are the ones who know the best about its operation, although they rarely share it, causing barriers in process improvement [4], demonstrating well the difficulties experienced in HEI, especially those of a public nature, given the frequent mobility of public administration workers.

Other difficulties are pointed out in the literature, namely the lack of experience in process improvement, coupled with a reduced number of human resources (HR) dedicated to quality and improvement [24], or even the compatibility of the IQAS implemented, with the requirements of other existing systems, such as A3ES, associated with the Portuguese context [5].

One of the advantages associated with the use of the EFQM model in HEI is its very nature, where, according to [14], it can focus on the organization’s "key clients", while meeting the current needs and stakeholders.

To do this, the model uses a series of appropriate indicators to monitor the performance of the organization and its various processes, well as the complementarity of the improvement to be performed with benchmarking actions, whether internal, either externally.

Another advantage is its certification, where, according to [25], it allows to certify the quality of the management practiced in the HEI, increasing the levels of efficiency and effectiveness through the allocation of resources, allowing the same recognition at both national and international levels between HEI and the companies with which they are integrated as partners.

2.4 Main models used

The accumulated experience with successful examples, coupled with the industry’s decades-long development of the "Quality Management" of its products/services, served as an incentive, and in a way, as a reference, for some HEI, felt the need to rely on quality management models from the industry to certify their services [26].

In this context and considering that HEI must implement reference models for their IQAS, the universally accepted models include the European Foundation for Quality Model of Excellence, the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG), the European Association for Quality Assurance in Higher Education (ENQA), the Balanced ScoreCard (BSC) and the International Organization for Standardization (ISO) standards.

The last models mentioned here are adopted from the industry, and lately, and in the perspective of continuous improvement, the focus on TQM ^[4]-inspired approaches, have increasingly assumed a greater relevance in the HEI universe [27].

However, given the complexity of HEI due to the service provided and the stakeholders involved, it is not easy to adopt and implement systems-based IQAS (TQM), although is a path that can be followed by HEI with the objective of continuously improving the quality of the service provided [28], which comes to the TQM philosophy, and in particular the model EFQM ^[5].

2.5 The EFQM model

The EFQM Excellence Model was created in 1992 to assist organizations in Europe and aims to establish a quality management system that allows the evaluation of organizations with the objective of continuously improving their performance, aiming at the attribution of the European Quality Award (EQA). Currently, the model is used by many organizations from different economic sectors, namely banks, insurance companies, oil companies, energy companies, health, schools, universities, etc.

The evaluation of the organization is performed according to a set of criteria and subcriteria, which are assigned a pre-established score [29]. The principles of the model are based on 8 fundamental concepts, which allow any organization to achieve

"Excellence" in a sustained way, and to establish a common language among managers [30], which are:

• Add Value to Clients;

• Building a Sustainable Future;

• Develop Organizational Capacity;

• Take advantage of Creativity and Innovation;

• Leading with Vision, Inspiration and Integrity;

• Manage with Agility;

• Succeed Through Talent of People;

• Maintain Outstanding Results.

Based on these concepts, the model proposes to the organizations the use of nine criteria, to analyze the relations of cause and effect, namely what the organization "does" through the means that it has, and what they "obtain" in matter [29], defining the EFQM 2013 model (Fig. 1).

Figure 1

The 9 criteria of the EFQM 2013 Model [6]

Several examples of the application of TQM models, namely in the context of the application of the EFQM model, or based on it, can be found on literature, some still in progress, others already completed, some partially implemented in some services (e.g. [31]), and others fully implemented, such as the DAETE project from the University of Porto (Portugal) [25]. This is a project funded by the European Commission and the US under the Atlantis program, which aims to develop various tools based on the EFQM model and applied in the context of HEI.

To this end, several self-assessment tests were carried out, covering 42 HEI in Europe, the USA and China, and the process was later adopted by the Association of Continuing Engineering Education as a tool for evaluating the quality of management at a global level [25].

Other studies have been developed, namely the adaptation of the EFQM model by the University of Sheffield in Hallam [32], or the study by [33] on the implementation of the EFQM model in the analysis and improvement of processes at the University of Firenze in Italy.

2.6 Initial problem

Through previous findings, the central question that defines the problem, created around on this study, can be defined as:

How can the EFQM model be implemented in an organization with the characteristics of an HEI?

In order to contribute with answers to the main question, the EFQM implementation was evaluated in a Portuguese IES, whose IQAS, is being elaborated based on EFQM model.

Based on what was referred before, several questions have arisen, which will enable us to answer the main question mentioned above, namely:

• What are the difficulties found in designing an IQAS based on EFQM model, to satisfy its different stakeholders, and given their different perceptions about the concept of ’quality’?

• How the implemented IQAS can frame the strategic guidelines to be defined (or existing already) by the organization, as well as any existing evaluation systems / requirements to which the organization is subject (e.g. A3ES, Ordem dos Engenheiros, etc.)?

• What are the possible advantages, that can be achieved through the implementation of an EFQM model into a HEI, attending the different stakeholders needs?

2.7 Research objectives

The research objectives to be carried out, are:

• Analysis of possible advantages with the implementation of an IQAS according to the EFQM model, given its own organizational structure, as IES;

• Analysis of possible difficulties found, regarding the implementation of EFQM model, and seeking to provide alternatives for overcoming them;

• Contribution to the study and analysis of the feasibility in the application of the EFQM model in HEI, through the accomplishment of the present study;

3.1 Adopted methods to collect data used

About the techniques for collecting data/information, these are based on the following:

• Documentary analysis.

• Observation throughout the process of implementing the IQAS (EFQM) in the IES.

The first one was based on the documentary analysis of several documents assigned to the different functional areas that make up the HEI under study, namely Departmental Areas assigned to the different courses taught, services (e.g. Financial, Human Resources, Procurement), complementary units (e.g. Library, Informatics), offices (e.g. Audit and Quality, Communication, Accounting & Heritage), Laboratories and R & D Centers.

The documentary analysis also focused on the following documents addressed to the management, as well as the entire HEI in general, namely:

• Various Regulations associated with the governing bodies of the institution (e.g. Supervisory Board, Technical-Scientific Council, Management Council)

• Annual Activity Report

• Annual Report of Accounts

• Annual budget

• Evaluation and Accountability Framework (QUAR) of the HEI under study

The second one, was based on the collection of data from the Office of Audit and Quality, responsible for the proposal for the implementation of the IQAS in the IES under study, as well as the consultation of the different

human resources, affecting the several functional areas mentioned above (example on Fig. 3), as well as the other elements belonging to the institution’s governing bodies.

Figure 2

Stakeholders diversity regarding the HEI studied

Figure 3

Example of relationship between functional areas and the stakeholder’s diversity regarding an HEI

3.2 Methodology adopted

In order to try to answer the questions, mentioned above, it will used the case study methodology, since it is the most appropriate strategy in answering the questions posed in the "how" or "why" research [35, 36].

According to the same author, the indicated methodology allows us to define an empirical approach that seeks to investigate a current phenomenon inserted in a real context, particularly appropriated when the boundaries between the phenomenon and the context are not clearly evident, allowing therefore, the construction of a theory, not only from the literature review, but also as a result of empirical observations or actual experiences that may result in both qualitative and quantitative research.

In this sense, this research is based on an inductive logic, since the possible confirmation of the applicability of the EFQM model in the IES under study, does not constitute evidence by itself, to be applicable to other HEI, but may contribute to the analysis and discussion of its viability [37].

3.3 Introduction to Case Study

Considering the above, the case study presented here, refers to a School of Engineering in Portugal, composed of about 4000 students, 483 Teachers and 116 non-teaching staff elements.

Like the generality of HEI, this IES also aims to satisfy a diversity of stakeholders that are part of it (Fig. 2).

If, at the level of the organizational aspect, and taking this IES as an example, it is desired to make a correspondence between the different functional areas and the associated stakeholders, it is verified that there is a diversity of stakeholders with different perceptions and quality requirements, ranging from according to the functional area to which they relate (Fig. 3).

Observing Fig. 3, the organization can be subdivided into small models of self-evaluation, which corresponds to subdividing the IQAS into parts according to their functional area, although related to each other.

According to [37, 38, 39], an HEI can be evaluated in three main areas: teaching, research and services.

This study will be focused in the services context, where it will be studied the feasibility of the IQAS implementation, based on EFQM model, by using a HEI as a case study.

4.1 Advantages found within model implementation

• Compatibility between the implemented IQAS, the Strategic Plan and the Assessment and Accountability Framework (QUAR) of the organization.

One of the main advantages observed with the implementation of the EFQM model, and which is clearly shown in the School Strategic Plan, is the focus of the model on the results obtained, resulting from the processes developed / managed, allowing the realization of an appropriate alignment between the school PE design form and the EFQM model, as well as the organization’s QUAR, due to the unfolding of its criteria and subcriteria in "means" and "results". This evidence is clear in the EP of the institution, by deploying the strategy developed in "strategic axes", by establishing the "operational objectives" (as a way of implementing the axis) and finalizing in the "actions" and "expected results" , monitored and validated through indicators associated with predefined objectives, and according to what is established in QUAR ([18] and [40]).

• Creation of "quality groups" for the continuous improvement of processes

The EFQM model allows the creation of quality groups in each functional area (also often referred to in the "improvement commissions" literature) to develop/reshape their processes under continuous improvement under the EFQM model. In the case of this school, the commissions cover each functional area and within the framework of a TQM approach, and it is also planned to create regular meetings between functional areas with a view to promoting internal benchmarking of the organization, to disseminate best practices between areas functional.

Therefore, it is expected that employees will be more involved in the continuous improvement of processes, together with the further development of industrial relations in the field of teamwork.

4.2 Difficulties found within model implementation

Among the difficulties found within model implementation, and in terms of impact to the organization, we can referrer:

• Lack of management tools (in the school) that act as an EFQM support models

This school, like all HEI, does not have plans, supported by management techniques, that are enough evidence to satisfy certain subcriteria within the framework of the EFQM model, given the differences in the techniques of public and private administration, the latter being, the initial purpose of the EFQM model.

Nowadays, and given the greater administrative autonomy from HEI in relation to the Government, and to attend the greater demands from it in terms of efficiency and effectiveness, there is a new "phase of transition", based on reduction of the HEI’s management dependence, in relation to the public administration, as well as from adopting even more, new management techniques from private administration (e.g. Balance Score Card, SWOT Analysis, among others).

In this sense, and since many HEI are still in this "transition phase", this fact constitutes a barrier in the implementation of an IQAS under the EFQM model.

• Resistance/barriers to the change

This difficulty is strongly related to the previous one, since it is essentially related to the use of models, supported by private management techniques.

As the training and experience of many employees is based on public administration procedures, changing some of these procedures when implementing the EFQM entails the assimilation of new skills, such as using the Balance Score Card, or managing the Social Responsibility, recently implemented in the organization [17], and as such, there is some inertia in its adoption, which causes some delay in the implementation process of the EFQM model, namely at the level of self-evaluation processes.

Another aspect is the assimilation of minimum skills by the employees of the organization, with a view to a broader understanding of the model, thus allowing more effective self-evaluation.

4.3 Ways to overcome the difficulties found

• More appropriate and timely organization and planning in the EFQM implementation

Likely the EFQM implementation in private organizations, also in public organizations, it is of special relevance an adequate planning, which prevents in a timely way the team of implementation of the IQAS, about the possible requirements for the elaboration of the IQAS. A team consisting of external / internal elements with technical skills of private management and experience in the implementation of the EFQM is also required, to work with elements with public administration skills, working together with Quality Managers and another representative element from organization.

• Provision of EFQM training and regular meetings to monitor its implementation

The previous disclosure of the intention by the leadership bodies, would allow a first approximation of the employees with the EFQM model and its specificities. In that sense, and at a later stage, employees would be offered an introductory training course, followed by a training plan, appropriated to each functional area, to assimilate the skills developed in the management techniques used in the organizational framework on behalf of the EFQM model. Such solution would reduce the time of adaptation of employees to new procedures, ensuring a better execution of the model after its implementation.