46

January 2010

  • Send
  • Subscribe

OPINION

Guide for the assessment/accreditation of external university centres

Jaume Sarramona López - Full professor of Pedagogy of Autonomous University of Barcelona (UAB)

One of the numerous tasks entrusted to the Agència per a la Qualitat del Sistema Universitari de Catalunya is to assess attached university centres offering degrees that are recognised by Catalan universities. It has been responsible for this function since 2008 and it has set up a specific protocol of an eminently qualitative nature to do so. 

However, in 2008 AQU Catalunya was entrusted with assessing/accrediting two universities from Latin America, specifically the Faculty of Law of San Martín de Porres University in Lima (Peru) and the Regional University Centre of the University of Guadalajara (Mexico). Indeed, it had already carried out an assessment of the six studies offered at this latter institution in 2008 and had published the respective report. In order to conduct the accreditation requested, a specific instrument was drawn up which incorporates European higher education quality standards from the perspective of the European Higher Education Area whilst taking into consideration the circumstances and traditions of university assessment in Latin America. To this end, the management of AQU Catalunya issued a resolution on 19 January 2009 commissioning me to draw up a pilot methodology for the assessment and recognition (accreditation) of the quality of universities outside the Catalan university system.

The instrument prepared combines a tradition based highly on quantitative aspects, typically applied in Anglo Saxon countries and in Latin America, with the highly qualitative tradition of the instruments applied thus far in Europe. This makes assessment tasks easier for committees and experts, since the methodology prepared makes it possible to specify the level (on a pre-determined scale) at which the institution assessed is situated for each of the aspects examined. It also makes it possible to determine qualitative criteria and clarifications prior to issuing an overall judgment on the assessment. The qualitative aspect is particularly significant when it comes to putting forward suggestions for improvement. The instrument comprises the following 8 sections:

1. QUALITY OF DEGREES AND PROGRAMMES
2. TEACHING STAFF
3. LEARNING RESOURCES
4. LEARNING ASSESSMENT
5. INFORMATION
6. RELATIONS BETWEEN THE INSTITUTION AND ITS SETTING AND SOCIETY AS A WHOLE
7. RESEARCH
8. GOVERNANCE

Each section includes several indicators making a total of 68, incorporating all European standards for internal quality assurance of higher education institutions, in addition to others considered important for the general quality of an institution or university centre drawn from the experience gathered from assessments of attached centres. Nonetheless, it is necessary to point out that section six on relations between the institution and its setting and society as a whole is drawn up with the circumstances of Latin America specifically borne in mind since it is a context where universities establish intimate ties with their settings and they actively cooperate in addressing any social shortcomings.

The assessment for each indicator is carried out according to the following five-point scale based on the extent to which the indicator target is achieved:

0) Unattained
1) Poor
2) Fair
3) Good
4) Excellent

The quantitative value given can reach up to 272 points. A minimum acceptance level for the accreditation is set at 160 points; if the institution receives between 175 and 220 points the quality assessment is good; if the institution receives between 220 and 272 points the quality assessment is excellent.

This strictly quantitative perspective is clarified by the fact that each section incorporates certain indicators deemed critical to the quality of higher education institutions. These indicators must be worthy of a minimum grade of three points in order for the institution to be able to show it has attained them in a qualified manner and, accordingly, for it to be accredited by AQU Catalunya. By way of example, certain indicators for some sections are shown: those considered to be essential are shown in italics and European indicators are marked with an asterisk.

1. QUALITY OF DEGREES AND PROGRAMMES

1.1. The existence of planning for degrees, subjects and programmes by the competent bodies (faculties, schools, departments, etc.). (*)
1.2. Specification of the learning goals in planning of programmes (in the form of professional skills or similar).
1.3. The range of teaching methods offered (classroom, partial classroom and partial distance learning, e-learning, etc.) in order to meet and attend to various needs and groups. (*)

2. TEACHING STAFF

2.2. Lecturers have the suitable knowledge and didactic techniques to be able to teach and perform all the responsibilities entailed. (*)
2.10. A teaching staff mobility system is in place.

4. LEARNING ASSESSMENT

4.4. Systematic monitoring is conducted on student progress and guidelines stemming from this are established to help students that fail the respective assessments. (*)
4.5. Regulations are in place in case of any incident in relation to assessments (absence, illness, etc.). (*)
4.7. Students may review exams and qualifications.


6. RELATIONS BETWEEN THE INSTITUTION AND ITS SETTING AND SOCIETY AS A WHOLE

6.1. The institution has ties with extra-university social bodies.
6.3. The institution has ties with public and private companies and bodies that hire its graduates.
6.8. The institution has an active environmental conservation programme: energy saving, waste treatment, etc.


7. RESEARCH

7.3. The institution takes part in research agreements and projects with other institutions, networks, etc., in order to conduct joint research.
7.4. Publications are made in prominent international scientific journals.
7.5. The institution offers PhD studies.

This instrument has successfully undergone a review by a committee of external assessors – the same committee that assessed the degrees of the CUCSUR in Guadalajara – and was subsequently subject to an empirical assessment at the same centre by means of the set-up of focus groups formed by all collectives involved in university assessment: directors, lecturers, students, non-teaching staff, graduates and employers, who gave certain considerations regarding the basic indicators and the draft of the text. Lastly, the proposed instrument was approved by the Specific Committee for the Quality Assessment of University Centres and Activities on 7 May 2009.

The methodology employed by AQU Catalunya in order to accredit a university centre in Latin America is the same as the methodology applied in general to assess universities in Catalonia. The institution subject to accreditation initially undergoes an internal assessment using the aforementioned instrument. This assessment must be carried out by the internal committees compiling evidence as required in order to justify the mark given for each indicator. The assessment of indicators for each section is supplemented by an overall judgment made regarding the specific section to highlight strengths and weaknesses, particularly focussing on those indicators deemed to be essential.

Following the self-assessment and the respective report, an external assessment is conducted by a committee of experts hired specifically by AQU Catalunya. The committee will be formed by assessors from Catalonia and the country in which the institution under assessment is located. They will conduct an on site analysis of the evidence provided and they will hold interviews with groups from both within and outside the institution. The purpose of these meetings is to conduct a more in-depth analysis of the indicators closest to these groups. The external assessors will conclude their duties by producing an overall report, in parallel to the internal one, which will include a proposal for the improvement of those areas that have scope for it, regardless of whether accreditation is given or not, focussing especially on those indicators deemed to be most important for the accreditation whilst taking into consideration the remainder. This report is sent to the institution assessed to allow remarks to be made and statements to be given prior to it being submitted to the AQU Catalunya accreditation committee. Another important step in the process conducted by the external committee before leaving the institution under assessment is to issue a verbal report summarising the subsequent written report, in order to place an initial valuation of the institution on record.

The instrument we have been referring to has proven to be useful to assessing the two institutions mentioned at the start: a law faculty in Lima (Peru) and a regional centre in Jalisco State (Mexico). The application of the instrument has shown that there is a need to adjust linguistic expression in accordance with the manner in which it is used in each country, though the overall assessment potential offered by the instrument has clearly been shown, making it worthy of its acceptance by those assessed thus far.

ENQA EQAR ISO

Generalitat de Catalunya

Via Laietana, 28, 5a planta 08003 Barcelona. Spain. Tel.: +34 93 268 89 50

© 2010 AQU Catalunya - Legal number B-21.910-2008