ShareKey concepts
Firstly, it can be said that it is unanimous acceptance that the scientific practice of evaluation consists of obtaining evidence (objective information of a quantitative and qualitative nature) in a systematic way to provide information for some type of decision-making process.
The different ways in which study programmes are assessed depends, on the one hand, on the tradition of evaluation, in some way formal, of higher education systems and, on the other, of the adoption of certain models that have become consolidated through critical thinking and evaluative research.
This section gives an overview of the different perspectives in terms of the various categories of institutional review in higher education.
- According to the objective of the review
- According to the review methodology
- According to the agents of the review
- According to the type of evidence that the review is based on
- According to when the review is made
- The evaluation model used by AQU Catalunya in the VSMA framework
According to the objective of the review
Quality assurance for accountability, characterised by external locus of control and associated with centralised administrative structures and external auditors measuring quantitative indicators of success.
Quality assurance for enhancement, characterised by an internal locus of control and associated with facilitative administrative structures that use peer review to assess more qualitative indicators of success.
Up
According to the review methodology
Accreditation. Accreditation is recognition of the status and legitimacy of an institution, programme or learning module. It is a process whereby a programme of study or institution provides information on its activity and outcomes to an external panel, which independently evaluates this information and then issues a public judgment based on specified standards concerning the institution or programme's value and quality.
The emphasis in accreditation is typically on the inputs as an assurance of the quality of the outputs, with the focus on effectiveness and a high level of summative assessment.
Programme review and evaluation. Unlike accreditation, this is not a comparison with pre-set criteria, but merely seeks to describe and assess the background of the unit or programme, its current performance and prospective plans. It is a process that analyses the quality of the learning process, including aspects such as teaching and pedagogical approaches. A review that is not the consequence of internal motivation will be difficult to implement.
Audit. This is a process for checking that procedures for quality assurance have been set in place. A quality audit checks the extent to which an institution or programme is achieving its own explicit or implicit objectives using established processes. It also checks the suitability of the planned quality procedures in relation to the stated objectives, conformity of the actual quality activities with the plans; and the effectiveness of the activities in achieving the stated objectives.
Up
According to the agents of the review
Internal dimension. Responsibility for the initiative and implementation lies with the higher education institution itself. Key factors for success are leadership, involvement and organisation of the process.
External dimension. Responsibility for the implementation lies with external bodies (outside of the university). The quality of the review panels, over and above the technology (guides and protocols) made available to them, is the determining factor for good review practice.
Up
According to the type of evidence that the review is based on
Evaluation based on performance indicators. The value judgment that is issued is a direct consequence of the confirmation of a certain quantitative value or the presence or absence of a series of characteristics in the object being evaluated. In general, there tends to be a certain degree of reductionism in the identification of indicators (ease, indicator reliability, potential for statistical analysis), and also their association with an accountability-based evaluation.
Evaluation based on expert judgments. This type of evaluation is sometimes confused with a lack of quantitative information (indicators), as if an expert's judgment was based on qualitative inference (the clinical eye of the reviewer). Its real meaning is that data (quantitative and qualitative) are contextualised, interpreted and assessed (within frames of reference) by reviewers. Any automatic interpretation of data is excluded from the judgment.
Up
According to when the review is made
Ex-ante model. Within the context of Europe in general, and Spain in particular, the accreditation (sui generis) of institutions and programmes, which is completed by governmental or official approval of the setting up of a university or curriculum leading to a recognised qualification, has always existed. The rationale on which this process is based is totally deductive and administrative: if the set requirements, standards or guidelines are complied with, then the quality is assured. The bodies responsible for these processes are governmental and guided by highly formal procedures with very little transparency.
Ex-post model. Common in higher education systems with a long tradition of autonomy in the provision, design and development of study programmes. This is generally linked to either accreditation or the evaluation and review of programmes and therefore calls for the completion of a time cycle in which the programme has been through all of the various stages of evaluation.
Up
The evaluation model used by AQU Catalunya in the VSMA framework
The current regulatory framework (Royal Decree 1393/2007, 29 October) clearly identifies a series of characteristics that the review model used by QA agencies must comply with, although it does allow for others. AQU Catalunya's long and extensive experience in institutional review (as can be seen from its past record) has ultimately led to the adoption of the model being presented here, which is characterised by:
- The integration of targeted objectives into quality enhancement and accountability.
- The combining of audit, programme review and accreditation methodologies.
- The complementing of the internal and external dimensions of the review.
- Integration of indicator-based evidence with expert judgments.
- • Consideration being given to the real time continuum as regard the time when the evaluation is carried out:
- Ex-ante validation (assessment) of the programme proposal. - Monitoring of the proposal's delivery. - Accreditation (ex-post assessment) of the study programme.
Up
10.3.2014