A bimonthly publication of AQU Catalunya

51

November 2010

Introduction

At present, European universities should be implementing reforms enabling them to become centres for knowledge management and, moreover, allowing them to act as the driving force for this knowledge to encourage greater levels of mobility and employment integration among their graduates.

One of the foundations of this reform being implemented in universities is to set up an internal quality assurance system (IQAS) making it possible to keep a check on processes whilst ensuring continuous improvement. Such an IQAS needs to be sufficiently stringent to report on how processes operate and the extent to which the goals set have been reached, while being sufficiently flexible to adapt to the constantly changing, competitive and globalised setting in which all universities are immersed. Moreover, the IQAS must also meet European standards.

Accordingly, AQU Catalunya has prepared and launched several programmes with the key aim of providing guidance and advice to universities in adapting to these changes. As of 2007 ex ante assessments have been conducted on the IQAS design within the framework of the AUDIT programme. These assessments also incorporate a review on the university's IQAS design carried out by a committee of experts following a protocol based on European standards. This makes it possible to gather useful recommendations for improving the IQAS and its subsequent implementation.

As a result of this experience, in 2009 the Suport programme was launched with the main objective of enabling the members of the technical units for quality and all individuals involved in quality issues to gain a first-hand acquaintance of how European universities are addressing the issues that are currently most important to them. For this reason, AQU Catalunya sought to identify the European universities that have best addressed these concerns.

Project description

There are two aspects operating within internal university quality assurance systems. An internal aspect whereby the IQAS must consider the characteristics and organisation of the institution and use information generated by the IQAS in order to efficiently and effectively monitor the university's learning programmes. Likewise, there is an external aspect wherein the IQAS is required to be as flexible as possible adapting to social and economic changes as well as the needs brought about by globalisation.

Indeed, setting up channels for communication among the various universities is a strategic matter to ensure that best practices flow swiftly and smoothly. The means used to tackle certain circumstances that many institutions may face in common could provide a source of inspiration and innovation for other institutions. It is consequently important to be familiar with best practices which, based on a prior adaptation to the characteristics of the university and its environment, make it possible to address circumstances that may currently be of concern to the individuals in charge of the quality units with a certain degree of success.

In line with these premises, within the framework of the Suport programme AQU Catalunya embarked on a study of European universities that were providing what could be considered as benchmark quality management models. In order to conduct this study, advice was sought from a host of European quality agencies:

• Quality Assurance Agency for Higher Education, QAA 
• Danish Evaluation Institute, EVA 
• Norwegian Agency for Quality Assurance in Education, NOKUT 
• Swedish National Agency for Higher Education, HSV 
• Accreditation Organisation of the Netherlands and Flanders, NVAO
• Finnish Higher Education Evaluation Council, FINHEEC
• Evaluationsagentur Baden-Württemberg, EVALAG 

In order to identify the benchmark universities, Catalan universities were asked what their main issues of interest were and they can be grouped under the following themes:

• Ensuring stakeholder participation: specific training in quality for stakeholders, advice and participation in quality management processes. 
• Becoming acquainted with how quality management units are organised and what duties they have been assigned.
• Becoming acquainted with how assessment processes are conducted on institutions and education programmes.
• Becoming acquainted with how information systems are organised and operate. 
• Becoming acquainted with student assessment mechanisms.

In order to make the first selection of universities, the IQAS of the universities recommended were studied based on information on their IQAS as published on their websites. In addition, data has been compiled on the number of students, the type of organisation (centralised or federal), whether it is public or private and academic fields, among other areas. Ultimately, the following universities took part in the programme:

The chosen universities received a presentation of the Suport programme with details of its goals, what was expected from universities' participation in the programme, the main characteristics of the university that would be visiting them and the issues that most interested them. Subsequently, each of the Catalan universities determined in conjunction with their host university the details of their visit and, following the visit, they drew up a report describing the following areas: the activities carried out during the visit, the main aspects of the quality system and the best practices identified.

Identification of best practices

The concept of best practice must be considered in the context of the Catalan university system. The following best practices were considered as innovative and it may be feasible to implement them in the short- to mid-term making the adaptations needed to ensure they fit in with the circumstances and regulations applicable in Catalonia:

1. Assessment and mechanisms for teaching Suport

Assessment of teaching staff is an important issue with regard to students achieving the learning aims indicated in university's learning programmes. Therefore, the assessment system is a mechanism that has to enable the continual improvement of the teaching staff. Thus, the two practices presented here promote an assessment undertaken by teaching staff themselves.

In the first practice, the dean is responsible for the organisation and supervision in each institution, following the guidelines laid down by the Centre for Academic Practice. Teaching staff, those making the assessments and those under assessment arrange a prior meeting regarding the date the observation is to be undertaken, what type of session it will be, the level of the students, teaching aims and the session's intended outcomes, potential difficulties, and any other relevant matters that need to be focused on. Following the observation, the member of staff making the assessment should meet with the assessed member of staff to address issues that enable the exchange of information between one another as a response to the data gathered by the assessing member of staff, and who should, as a final outcome, propose alternative strategies and techniques considered necessary to achieve the development and improvement of the performance of the observed peer. This is considered to be good practice because firstly, it enables the exchange of experiences and knowledge between teachers, and secondly, because it is not considered as an examination system, but rather as an opportunity for improvement.

The second practice is based on the involvement of the teaching staff in the quality control management procedures, which is considered to be a very positive aspect as the achievement of a university's stated aims depends on their role in the learning process. In this case, teaching staff take part in the assessment through a questionnaire, the results of which are commented on and analysed by the staff themselves in special meetings in which they share experiences, and which provide a space where they can learn from one another. This process is considered to promote confidence and a broader knowledge amongst the teaching staff.

2. Involvement of students in quality management processes and student assessment

a. Involvement of students in quality management processes

The students are the central focus of any university quality system and so to assure their involvement is of the utmost importance. The good practice highlighted here is based on the training of student representatives especially those that take part in the programme committees. It seeks to promote students' knowledge of the university's structure, its outlooks and aims, the different roles within the university, and how the quality monitoring and control (internal and external mechanisms) is undertaken. The content of the course is disseminated through student focused communication channels. The advantages of this practice are that it encourages the integration of students in participatory bodies, promotes an awareness of what is expected from them with regard to quality management, and it favours the exchange of experiences and problems.

b. Student assessment

With regard to the assessment of students' acquired skills, two good practices have been identified.

• The first of these is the assessment of transversal skills. Firstly, the university systemises the skills by area and according to three training levels. Students will identify in what activity they acquired each of the skills and how they can demonstrate their acquisition of it. It should be noted that this applies to adult students with prior professional experience. The fact that these students have teaching materials from previous courses makes them aware of the skills they have acquired and how this has improved their professional portfolios and the jobs they can apply for. In addition to this, the university works with the students so that they can describe the acquired skills in an clear manner to employers. Like this, students recognise how the training they have received implies an added value that will enable them to establish or improve their position in the labour market.
• For the second practice, students have access to a guide created by the students themselves which explains the assessment process. At the end of each of the modules, the students taking part complete a satisfaction questionnaire in which they are requested to evaluate the module. The process is based on a series of standard questions but it also includes a space for the modules' tutors to insert three specific questions relevant to their own modules so that they can gather qualitative information. The module's tutors analyse this information to prepare and set in motion a suitable action plan. Any of the issues raised or the measures adopted are introduced during the annual assessment process. Each module's tutor is also responsible for guaranteeing that students are informed of the results of the analysis of their contributions. This is considered to be a good practice because it obliges the university to have a clear and comprehensible assessment system that is able to measure student progress.

3. Organisation of quality systems

The good practices in this area are related to the involvement of the different interest groups in the quality control system.

The first of these is based on the involvement of different centres for innovation in the creation of an Innovative Management Centre Network. To be included in this network, centres must fulfil certain criteria regarding commitment to quality management and on-going improvement, such as: having undertaken a self-assessment or introduced a quality control system according to the ISO model and having established the explicit commitment for this from the governing body. This involvement has been very positive and it is not without financial implications as the centre may have its budget increased as a result of its involvement and the outcomes obtained.

In addition to this, an information system has been designed that allows centres to manage a strategic plan, undertake day-to-day procedure management and measure other aspects linked to its daily activity. This tool enables centres to share and compare knowledge with one another.

The second good practice is related to the considerable amount of work that has been dedicated to the involvement of staff through activities such as the semi-professionalization of personnel who collaborate with quality teams, the attendance to seminars and training activities organised by the quality unit, and the participation in improvement teams for specific periods, amongst others.

The third good practice described here is based on the work undertaken by the programme committees, which include staff and students, as well as a director and vice-director of a programme and a secretary. In addition, there is an executive committee made up of a teacher, a student and a programme director. The role of these committees is: to present on a three-yearly basis proposals on the study plan in relation to the skills profile demanded by the professional market and also the expectations of the students; evaluate on an annual basis the quality of the teaching delivered and make recommendations regarding the organisation of the curriculum. An interesting perspective of this practice is that it systematises a triennial revision of a programme's content on the basis of an assessment by the professional sector. In addition, a student gives his or her point of view, providing a more critical view that is different to that of the academics. The student has the opportunity to describe the education he or she expects to receive. Finally, it is considered a good practice because the committee analyses every result of the programme.

The fourth good practice is related to the external assessment for specific programme modules or courses. The external evaluator has to monitor the programme and carry out an annual assessment report on the accomplishment of standards. The report has to include the recommendations and good practices that could help to improve the development of the study programme. This report is reviewed by the study programme director who has to respond to the main issues detected in the report and identify the improvement measures that will be implemented. It is considered a good practice because the external evaluators have a commitment contract with the university that implies a higher implication in their assessments.

The fifth good practice is related to the different stages of study programme assessment. The university establishes three different assessments: post-launch, annual and after a period of six years.

• El principal objectiu de la revisió de la posada en marxa és garantir que tot el que està relacionat amb el programa d'estudis s'adeqüi als objectius que cal assolir. Sovint, les accions de millora estan adreçades a redefinir els objectius del programa d'estudis i les expectatives dels estudiants.
• The annual review is internal. Its main aim is to check if the results achieved are in line with the study programme's objectives. Subsequently, the team responsible for the study programme has to design the improvement actions.  
• The six year review is an exhaustive study of the whole programme. It includes an assessment of aspects such as learning experience, learning outcomes, curriculum design, learning resources and the quality assurance system. The final report has two main sections: good practices and improvement areas.

4. Information systems

One of the major concerns is how to manage efficiently all the information generated by the quality management system. The information systems have been designed to gather all the information necessary for the follow-up of the study programmes. Thus, it is capable of providing strategic information to the management team. On the one hand, the data is processed in order to measure the strategic plan progress and on the other hand, the information system gives an integrated view of the whole university comparing all the schools and faculties.

5. Wider social participation

Universities in the process of becoming knowledge management centres should take into account society's needs. Accordingly, this good practice should include ways for society to be involved in the strategic plan.

There are several ways in which this can be done, for example, bringing university services in line with society's needs and focusing research on more applicable economic and social purposes. There are three different areas of work:

• Social interaction leadership: defining university responsibilities and functions, establishing terms of collaboration, developing a follow up and review system, analysing stakeholder groups.
• Strengthening community responsibility: emphasising the importance of higher education for society, encouraging the application of knowledge to society's needs. 
• University associations: strengthening a university's ties with business environments, involving the university in regional development, promoting knowledge exportation, bringing the university closer to adults through lifelong learning. 

Implementation of good practices

The Suport Programme has received a very satisfactory assessment by the universities that participated. This success is based in part on the programme's win-win character. The exchanges of experiences between universities have generated synergies among people involved in quality issues. Therefore, the Suport Programme has contributed to the strengthening of ties between quality management units across Europe.

For Catalan universities, the biggest impact on their day-to-day work is the opportunity to learn new ways of dealing with matters that most concern them. The majority of the good practices need to be adapted to each university's circumstances and requirements, but all of them were an inspiration.
The Suport Programme has provided experiences that have been the starting point for projects. Moreover, it has served to revise the different ways in which quality is assured and enhanced.

Projects and actions that have arisen from the Suport Programme: 

• Peer-to-peer assessment. One of the universities is planning to launch the design process for teaching assessment. Currently, the Quality Unit is designing the process, defining responsibilities and outcomes. Next year a pilot experience will begin in one programme. In the mid-term, the unit plans to apply this practice across the whole university.
• Student's involvement. The Quality Unit is designing a training activity that will be compulsory for all student representatives. It will be an optional activity for all other students, available to all study programmes and with a number of credits assigned. The content of this activity will be: basic knowledge about organization and university structure, awareness of quality management and the importance of follow-up actions and channels for participation.
• Prioritization of quality actions. The visit provided the Quality Unit with a wider range of criteria that help focus its efforts on fundamental aspects of quality enhancement. In particular, involving stakeholders in quality management issues by highlighting the impact of quality actions on their day-to-day work and demonstrating that process improvement in the mid-term can reduce the volume of work for the unit's members.
• Decentralization of quality management.After the visit, the Quality Unit is aware of the importance of engaging management teams in every school. It is particularly important to involve all stakeholders, management teams, academic staff and services staff. To this end, personnel from the Quality Unit are holding meetings with every school to inform them of what it is expected of them, their level of implication, how they can participate in Quality Management matters, responsibilities, etc. The Quality Unit is also launching a project to train coordinators of study programmes and academic staff in quality management issues.
• Simplifying the follow-up process.As a result of the visit, the Quality Unit is heading a project to simplify all follow-up processes, analysing which data and actions have added value to be collected and implemented.
• Re-designing processes in order to create a study programme. The Quality Unit is analysing stages, stakeholders, functions and responsibilities. Moreover, it is working to establish when information is confidential and when it falls in the public domain and therefore has to be published.
• To define a student data model.The Quality Unit is developing a project to study how to integrate all student data that is currently in different databases. The main purpose is to have a data base which integrates the courses, itineraries and personal data for all students. In addition, they are regulating access to this data, establishing who and how this data has to be treated.

Benefits for the host universities

At the end of the stays, cooperation was sought from the host universities to gather their opinions on the Suport programme. Interviews were therefore conducted posing questions on the degree of satisfaction with the scheme. All universities interviewed stated that they would be happy to repeat the experience.

In terms of positive aspects stemming from the experience, the parties agreed on the following points:

• The preparation of the visit was a vital moment of reflection as to why things were conducted in a specific manner. Also, the views of individuals who were not linked to the organisation regarding everyday or obvious aspects meant that they were reconsidered. 
• The flow of information and various forms of approaching a common problem were highly inspiring.
• The visit was considered a source of motivation for the team as it meant that the efforts undertaken had been duly acknowledged.