A bimonthly publication of AQU Catalunya

44

September 2009

The quality assurance of university studies is a challenge that has been taken up by the universities in Catalonia in a committed and responsible way. In order to successfully manage this challenge, the universities need policies and internal quality assurance systems (for the purposes of this document, referred to by the acronym IQAS) that are both formally established and publicly accessible.

The aim of the AUDIT programme being implemented jointly by ANECA, ACSUG and AQU Catalunya is, in the initial stage, to direct the design of IQAS used for university studies through the integration of all of the activities connected with the quality assurance of degree programmes. In a subsequent stage, the programme will watch over the implementation by institutes and faculties of IQAS that have been favourably evaluated. The agencies will then have to certify the IQAS once they have been implemented for a number of years, during which time sufficient data, indicators and evidence can be gathered concerning the programmes of study within the scope of the IQAS.

The AUDIT programme is structured according to a series of guidelines that principally conform to the ENQA Standards and guidelines for quality assurance, it complies with Royal Decree 1393/2007, concerning the regulation of recognised university degree programmes, and it is also linked to the VERIFICA Programme  (point 9 in appendix 1). In specific terms, the guidelines for the AUDIT programme are as follows:

• Quality policy and objectives. 
• Quality assurance of programmes of study.
• Student orientation and degree programmes.
• The quality assurance and enhancement of academic staff and teaching support.
• Management and improvement of physical resources and services and of the quality of administration and services staff.
• Analysis and use of outcomes.
• Publication of information on degree programmes.

The main objective of the evaluation of IQAS designs is to assess how correctly the abovementioned guidelines are incorporated into quality assurance systems. The general criteria for evaluation also take consideration of the support of the university's governing bodies for IQAS and the specification of who exactly is responsible for their implementation, maintenance and continuous enhancement.

With the first call for applications of the AUDIT programme completed (the call made in 2007), now is a good moment to make an initial balance of AUDIT. The first call was made at the end of 2007 and it involved two stages, the sending and subsequent evaluation of the documentation connected with the IQAS. There was a representative participation in the call by both public and private-run universities in the Catalan higher education system, with eight (8) universities submitting IQAS for a total of 20 schools and faculties. Table 1 shows the list of the Catalan universities participating in the AUDIT programme, together with the participating schools and faculties.

Table 1. Universities, schools and faculties and type of IQAS model design in the 2007 call for applications of the AUDIT programme

The scope of the IQAS in the 2007 call was initially set at the school or faculty level, as to the university level. Two trends were evident in the design of IQAS models submitted: on the one hand, general models at the university level and, on the other, specific models at the faculty and school level (see table 1). The majority of Catalan universities chose general IQAS models at the university level (six universities), meaning the IQAS that was evaluated was the same for all of the university's schools/faculties that participated in the call, and these universities could probably have submitted just one IQAS with a wide enough scope to cover all of its schools and faculties. Only two universities submitted specific school/faculty level models in which, despite there being certain core aspects common to the whole university, each school/faculty adopted a different IQAS to others in the same university. The different characteristics justifying the choice by a university of a general university level IQAS model or a specific faculty/school one include the university's quality policy and goals, its organisational structure, the size of the institution, its course of development and the student profile.

Furthermore, there was a great diversity in the IQAS designs produced by Catalan universities in the 2007 call for applications of the AUDIT programme. This fact is considered to be highly positive as it confirms the active involvement of Catalan universities in this project and shows that a very careful process of reflection was made during the design stage of the IQAS and that is highly representative of the actual situation in the university/school/faculty.

Out of the twenty (20) IQAS submitted in the 2007 call for applications, sixteen (16) were given a positive evaluation for their design, either in the first evaluation stage or after having included any modifications suggested by the review committees. From the results of the pre-evaluation stage, there had been a certain imbalance in the way that the guidelines for the IQAS were developed, as can be seen from table 2.

Table 2. Results of the evaluation process in the pre-evaluation stage

The development of certain guidelines was either sufficient or satisfactory in the majority of cases, such as the guidelines connected with the management of physical resources; services, administration and services staff; and the publication of information. Other guidelines needed more development than was actually expressed in the documentation submitted, as in the case of guidelines associated with actions that encourage student learning and the analysis and use of outcomes. All in all, once the modifications and improvements recommended by the review panels had been made, all of the guidelines were assessed as being sufficient in the majority of cases, as can be seen from table 3. 

Table 3. Results of the evaluation process in the final evaluation stage (subsequent to any modifications made to the submitted documentation)

The AUDIT programme allows the universities to reflect and find the balance between processes that are already developed and the key guidelines for the quality assurance of university programmes of study, within the framework of each university's own strategic plan. In certain cases, the correspondence between the processes and the guidelines was direct and unequivocal. In others, the university had to arrange its process map to ensure that it covered all of the elements included in the AUDIT guidelines.

As a result of suggestions received from the stakeholders (universities, reviewers and agencies) through the satisfaction surveys and active dialogue with AQU Catalunya, certain changes were made to the supporting documentation for the evaluation process prior to the start of the new call for applications, which led to adjustments to the criteria and the evaluation protocol. AQU Catalunya also developed an extranet as a computer platform for the entire process, and modifications were also made to the composition of the review panels so as to admit the participation of students.

The positive response by the universities to the current call for applications (2009) adds to the consolidation of the AUDIT programme in the higher education system in Catalonia. Six universities have submitted IQAS in the current call, meaning that ten out of the twelve universities in the Catalan higher education system have participated in one or other of the two calls. The new call also provides for the possibility of exporting the positive results of the evaluations of a university faculty or school's IQAS to other faculties/schools in the same university that commit to accepting the IQAS that has already been positively evaluated. This option covers the situation of universities that chose to develop a general IQAS model for all of their faculties and schools and that, up until the present time, have only been authorised to submit applications from a limited number of faculties and schools.

Lastly, AQU Catalunya wishes to express its satisfaction with the Catalan universities' extensive involvement and participation in the AUDIT programme. It is envisaged that this will be maintained throughout the new call for applications. Following the design and evaluation of the IQAS, the real challenge facing the higher education system in Catalonia will be for the implementation of internal quality assurance systems to make the monitoring of degree programmes much easier and faster and ensure the continuous enhancement of university studies.