ShareAQU Catalunya analyses the potential of artificial intelligence in the degree programme accreditation
This preliminary exploration of the use of artificial intelligence analyses how effective the application of this technology can be in predicting the results of university degree programme quality accreditation.
AQU Catalunya has analysed the effectiveness of artificial intelligence (AI) in a study that preliminary explores the use of this technology in the quality assessment processes of university degree programmes. Conducted at a propitious time because the higher education system has an increasing volume and variety of data and because quality assessment processes are moving towards a centre-based approach, the study focuses on the potential of AI in data prediction, classification and clustering, as machine learning algorithms can learn complex patterns and relationships between variables. As such, the study aims to assess whether simplified assessment tools can be constructed to provide a useful complement to the different actors involved.
A total of 1,235 (498 Bachelor's and 737 Master's degrees) were included in the study, which took into account all the accreditation protocols for AQU Catalunya's university Bachelor's and Master's degree programmes up to 2021, of which 70-80% obtained a pass mark and the remaining 20-30% obtained accredited with conditions or towards excellence. Each of the protocols has been assigned data from the indicators available to AQU Catalunya in order to analyse its adjustment with the actual results obtained.
Analysis is structured in two phases - the first is a descriptive analysis and the second applies machine learning techniques - raising conclusions such as, for example, that the correlation between the quantitative indicators analysed and the results of the accreditations awarded is low. However, the ECTS taught by PhD lecturers is the indicator most closely correlated with the results of the accreditation. Although this analysis is only intended as a starting point for reflection and exploration of these techniques for degree assessment, the study also concluded that the achievement of an accredited result toward excellence and a low level of student satisfaction is a situation that occurs in 3-5% of the assessments.
The report can be found in full aquí. AQU Catalunya's Quality Assurance Department held a workshop with technicians to discuss the project's quantitative analysis results with a view to promoting internal reflection. Furthermore, the workshop also sought to gather the qualitative opinion of Senior technologists regarding the indicators used in accreditation and to see how they matched up with the quantitative results obtained. The report has been made by AQU technicians Dani Torrents Vilà and Núria Mancho Fora.