Share2023 Employment Outcomes Survey to use mixed method: online and phone
One of the methodological learnings derived from the seven editions of the Employment Outcomes Survey coordinated by AQU Catalunya is that it is increasingly difficult to obtain answers by telephone in a markedly young group. This, together with the increase in international mobility (both entry and exit) of the Catalan university system, means the use of the telephone as the only method of data collection should be reconsidered in order to guarantee and improve the scope and impact of this survey.
In this sense, and following the model of peer studies in Italy and the United Kingdom, the 2023 Employment Outcomes Survey will move towards a mixed method: online (CAWI) and phone (CATI). To advise the agency on this challenge, AQU Catalunya has set up the Working Group on the Adaptation of the University Employment Outcomes Survey, with the aim of analysing the advantages and possible drawbacks of the mixed model, as well as the changes required in the questionnaires and the possible impact on comparability with the results of previous waves. The experts in the group have issued a report detailing what measures can be taken to improve the efficiency of the mixed data collection that the study is implementing this year, which will be published next year.
The members on this task force are:
- Jaume Garcia (President), full professor in the Department of Economics and Business at Pompeu Fabra University and former president of the National Statistics Institute
- Raul Ramos, Professor of Applied Economics at the University of Barcelona
- Jasmina Berbegal-Mirabent, professor at the Technical University of Catalonia (UPC)
- Jorge Cátedra, head of projects at the Institute of Regional and Metropolitan Studies of Barcelona
- Sergio Álvarez, statistical quality technician at the Statistical Institute of Catalonia
Some of the main conclusions contained in the report issued by the experts in the Working Group on the Adaptation of the University Employment Outcomes Survey are:
- The main advantage of a mixed over single-mode information gathering method is that multi-channel methods allow you to design more diverse communication, contact, and information-gathering strategies and reduce coverage error, non-response, the response order effect, acquiescence, and social desirability bias.
- Furthermore, due to the fact that there is no interviewer, electronic questionnaires increase the error of non-differentiation and arbitrary response. Accepting the status quo in the answers, the “don't know/no answer” response rate, or the effects of the questionnaire design could also be a disadvantage in the new model. However, most of these errors can be minimised with a better questionnaire design.
- One possible action that could facilitate greater cooperation among respondents is the use of incentives and thus reduce the risk of methodological change. It seems necessary to make a prior cost-effectiveness assessment of this type of action.
- Previous international experience, such as in Australia, shows how job placement surveys provide very comprehensive information on companies where graduates work. This experience could be especially useful in the context of the Employer Survey, which is currently in its third edition..
The change of collection method (CATI to CAWI CATI) in the Employment Outcomes Survey 2023 [Catalan]