elButlletí

A bimonthly publication of AQU Catalunya

57

November 2011

• 
• 

AQU Catalunya's Teaching Staff and Research section has been using a procedure since 2008 to analyse the level of satisfaction of applicants using satisfaction surveys. A questionnaire is sent once a year to everyone who has applied for a junior lecturer assessment report or the accreditation of research (for senior lecturers).

The results of the surveys are incorporated into the AQU's internal quality assurance system and referred to the Agency's Board. An extract of the results will be published shortly on the website for the first time.

The article sets out and presents the main results of the satisfaction survey of applicants who participated in the first call for applications made in 2010 for junior lecturer assessment reports and the accreditation of research.

Survey specifications

	Junior Lecturer	Senior Lecturer
Population	380	173
Responses received	168 (44%)	68 (39.3%)
Margin of error (95% confidence)	5.77%	9.47%
Methodology	Methodology Questionnaire sent by e-mail, with space for making comments
Period covered by the survey	25 February, 2011

 

Assessment of the level of satisfaction with the information received

Every year the Teaching Staff and Research section deals with around 1,200 in-person inquiries, 3,500 telephone calls and over 600 inquiries by e-mail. The figures show that the majority of people contact AQU Catalunya before making their application (67.5% in the case of junior lecturers and 53.7% of senior lecturers).

91.1% of all applicants for the junior lecturer assessment report in the survey were favourably impressed by the attention they received from the Teaching Staff and Research section. In the case of applicants for the accreditation of research, 86% were favourably impressed by the attention they received from AQU Catalunya. This high level of satisfaction with the information received coincides with the level established from the data in the 2008 survey.

Accessibility to the information on the AQU Catalunya website

Eligibility requirements

Applicants gave a rating of over 3.5 (on a scale from 1 to 5) to the accessibility of information on the Agency's website. The most highly assessed aspect was the standard application and CV forms.

The assessment by applicants using a scale from 1 (very inaccessible) to 5 (very accessible) gave the following data:

• 74% of users, in the case of junior lecturers, thought that the information referring to the eligibiligy requirements was clear and accessible on the website.
• In the case of senior lecturers, 72% made a positive assessment of the layout of the information on the website.

In subsequent calls for applications, the electronic processing of the assessment process will be upgraded and expanded. A study has been started to analyse the possibility of adapting the software so that an applicant's curriculum can be uploaded and notification of the results of the assessment sent on-line.

Clarity of information on the assessment criteria

48% of the applicants for a junior lecturer assessment report gave either a 4 or 5, with 5 being the highest score, for the clarity of information on the assessment criteria published on the website, while 23% gave this 3 out of 5. Around 30% considered that the information was not very clear.

1-to-5 rating scale where 1 is the minimum value and 5 is the maximum value

With regard to applicants for the accreditation of research, the figures were similar, with 42% giving either a 4 or 5 for the clarity of information on the assessment criteria, 30% gave it a 3 and the remaining 28% either a 1 or a 2. These figures are similar to those for 2008, when the overall score for this was 3 for junior lecturers and 2.83 for senior lecturers.

1-to-5 rating scale where 1 is the minimum value and 5 is the maximum value

Suitability of the layout of the curriculum vitae (CV) for stating merits

81% of applicants for a junior lecturer assessment report considered that the CV structure used by AQU Catalunya was suitable for setting out their merits. For applicants for the accreditation of research, the percentage was 70%. These results are also similar to those for 2008. The possibility is now being studied of applicants using a standardized science CV format.

Composition of the review panels

78% of the applicants for a junior lecturer assessment report positively assessed the composition of the review panels. For applicants for the accreditation of research, the percentage was similar (76%). In 2008 the overall level of satisfaction regarding the composition of the review panels was 3 in the case of junior lecturers and 2.8 for senior lecturers.

1-to-5 rating scale where 1 is the minimum value and 5 is the maximum value

Barely satisfied with the assessment criteria

One of the key questions in the satisfaction survey referred to the appropriateness and level of requirement of the assessment criteria in relation to the merits associated with each criterion. As in 2008, the criteria were considered to be demanding yet appropriate. On a scale of 1 to 5, the average was 3.16 for junior lecturers (in 2008 it was 3.01) and 3.2 for senior lecturers (in 2008 it ws 2.95). This means that even though there were applicants who were not successful in their application for a report or accreditation, this section was given a pass rating.

1-to-5 rating scale where 1 is the minimum value and 5 is the maximum value

The grounds for unfavourable reports

Applicants who were unsuccessful in their application (they were unfavourably assessed) were asked to assess the reasons given, as this is a key element that can be used to guide enhancement strategies.

In the case of applicants for a junior lecturer assessment report, out of the forty-two (42) people who answered this question, 38% considered that the reason/s were clear, whereas slightly more than half were of the opinion that the grounds given were not very clear.

In the case of applicants for the accreditation of research, out of the 12 people who answered this question, one quarter of the total were of the opinion that the grounds given were clear, whereas the rest considered that they were not very clear.

As in the 2008 survey, these results underscore the need for improvements to be made regarding the clarity of reasoning expressed in unfavourable reports and accreditations. A protocol for good practices in giving grounds for unfavourable reports and accreditations has been approved by the CLiC and CAR Commissions this year, which is expected to improve the results in this section in surveys in the future.

Reasons for applying for a report or accreditation

A question was included in this survey regarding the reason why a junior lecturer report or accreditation of research was being applied for. While it is of course obvious that the main reason is to gain admission to university selection processes for contract teaching staff, it is interesting to note that a substantial percentage of the applicants wish to obtain external recognition of their academic and research career. In the case of applicants for a junior lecturer assessment report, this percentage was 32%, and for applicants for the accreditation of research it was 35.7%.

Lastly, in the open comments section, applicants for a junior lecturer assessment report stated that excessive importance is given to the fact that research publications must be in citation indexed journals, especially in the Social Sciences and Humanities.