A quarterly publication of AQU Catalunya

97

April 2020

AQU Catalunya's quality management system is certified to the international ISO 9001:2015 standard. As a consequence of this, and to ensure and guarantee a robust internal quality system, AQU Catalunya annually measures the satisfaction of all stakeholders in its activities.

The aim of this article is to describe the various mechanisms available to the section in AQU whose area of work is Teaching Staff and Research to gauge user satisfaction with the public service that it provides. More specifically, this refers to the assessment and accreditation of candidates to contract teaching staff positions at public universities in Catalonia who require a favourable preliminary report, in the category of tenure-eligible lecturers, and an accreditation of either research or advanced research for senior lecturers and professors, respectively.

The sources for measuring satisfaction are as follows:

• The compilation of complaints, acknowledgments, inquiries and suggestions made through the website mailbox and the Agency's teaching staff-related mailbox (professorat@aqu.cat). All complaints are dealt with as non-conformities whereas suggestions are appraised for inclusion, wherever possible, as enhancement proposals. Of note is the number of e.mails ( a total of 5,839) to professorat@aqu.cat that were replied to in 2019. Following an analysis of the volume and categories of all of the inquiries, actions were introduced to restructure, clarify and make information more accessible.
• The meta-evaluation reports drawn up by the chairpersons of the various specific committees where an assessment is made, among other aspects, of the independent role of the committees, the QA and assessment criteria, the relationship between the various review panels, and all strong and weak points and enhancement proposals for the review procedure. These meta-evaluation reports are shared with AQU's Governing Board.
• The surveys of applicants. Every year a survey is carried out of the applicants to the first calls for the accreditation of tenure-eligible lecturers, research accreditation and advanced research accreditation.

1st CALL IN 2019 FOR TENURE-ELIGIBLE LECTURERS 2019

 

Population: 363	Sample: 127	Responses: 34,9%	Sampling error: 7,2%

 

1.ª CONVOCATORIA ACREDITACIÓN DE INVESTIGACIÓN Y DE INVESTIGACIÓN AVANZADA 2019

 

Population: 434	Sample: 203	Responses: 46,8%	Sampling error: 5,1%

 

One strong point that stands out in both surveys is the high degree of satisfaction of applicants with the information received from Teaching Staff and Research at AQU (around 90% of tenure-eligible lecturers and 89.63% of senior lecturers and professors). Tenure-eligible lecturers also rated very highly the accessibility to information on AQU Catalunya's website, in particular to the application form, requisite supporting documentation and the eligibility requirements.

As far as the weak points are concerned, of note, on the one hand, was the fact that a large number of survey respondents who did not pass the assessment commented that the rationale given for an unfavourable resolution report was unclear. In this regard, the Research Assessment Committee approved the application of a best practices protocol for the drawing up and reasoning of unfavourable reports. 56% of the tenure-eligible lecturers who responded to the survey also rated the time limits for accreditation decisions as 1-2 out of 5. In order to improve this point, AQU Catalunya has implemented the project to comprehensively digitalise all quality assurance procedures dealing with teaching staff so as to modernise and simplify the decision and resolution process.

• Surveys undertaken of experts on QA review panels dealing with teaching staff. A survey is carried out every two years, with one being carried out in 2020 of all the reviewers who took part in accreditation applications procedures in 2019.

SURVEY OF REVIEWERS DEALING WITH TEACHING STAFF: 2019

 

Population: 285	Sample: 130	Responses: 45,61%	Sampling error: 6,5%

 

One strong point here is the overall rating for the QA review procedure of 8/10. More specifically, the mostly highly rated items were the support received from AQU Catalunya, the extranet as a review instrument and the bibliometric review of section B.1 in the CV, Articles in indexed journals, to ensure the accuracy and truthfulness of data disclosed. These variables were rated at between 4 and 5 (out of 5).

As for weak points, firstly, the workload was rated on average at 3.4 (out of 5). In this regard, actions are being introduced to widen the pool of reviewers in accordance with the Agency's QA standards. Secondly, the remuneration for each case reviewed was given the lowest rating of 2.5 (out of 5). AQU Catalunya is fully aware of this problem and is currently working to approve remuneration that is much more in accordance with the level of expertise of our reviewers. In spite of the rating given in relation to this section, our reviewers as a whole continue to demonstrate a strong commitment to quality enhancement in the university and higher education system in Catalonia.

The analysis of the satisfaction findings is presented to AQU's Governing Board, the Research Assessment Committee and the Internal Quality Committee. In addition, any enhancement proposals that stem from this are included in the internal QA system so they can be prioritised and incorporated. The knowledge and understanding of the opinions of our accreditation applicants, as well as the chairpersons and members of the specific committees involved in research assessment and accreditation, is thus of fundamental importance to us at AQU's Teaching Staff and Research section for improving and enhancing the management of quality assurance procedures.