A quarterly publication of AQU Catalunya

98

July 2020

The on-site visit consisting of a meeting of experts and members of higher education institutions (HEIs) – as part of a peer review process – is still regarded as the core element in assessment procedures on study programmes, quality management systems or research projects. The corona pandemic has posed a particular challenge to these procedures: can virtual “on-site visits” (video conferences) replace or absorb the added value of face-to-face encounters? We have explored the possibilities and limitations in recent months and wish to summarise some of them.

Since mid-March 2020, evalag has offered HEIs the possibility of replacing on-site visits in ongoing assessment procedures with video conferencing. Most of our partners have used this option to move their assessment procedures forward and, in general, we have received positive feedback from both HEIs and experts.

On-site visits regularly follow a tight, busy schedule due to the fact that experts travel long distances. Virtual meetings reduce travel time and costs and consequently allow for more flexible schedules. Initial expert rounds – usually held on the day before the visit – may therefore be held one week earlier to allow time to prepare subsequent talks with HEI members in more detail. Talks with HEI members may also be limited to half days and spread over three or more days, depending on the overall workload of the experts. However, spreading site visits also carries the risk of losing focus of the procedure.

The available conferencing systems (e.g., BigBlueButton, Teams, Adobe Connect, Jitsi, Webex and Zoom) differ mainly according to their performance with large group meetings. (Self-hosted) open source solutions promise better data security but depend on the available hardware performance and tend to provide lower video and audio quality, whereas proprietary solutions usually have enough bandwidth available and therefore promise better stability. We regularly use additional features like waiting rooms, chats, breakout rooms, instant feedback and screen sharing. Some solutions provide more or other features (such as the whiteboard) and we would also advise keeping these possibilities in mind when choosing a system. According to technical and security requirements, we highly recommend following the latest security notes while selecting a tool and keeping yourself up-to-date.

Virtual on-site visits require different preparation with regard to the schedule, organisation and content. The discussion rounds, for example, should be prepared with a clearly structured agenda in advance. Since video conferences are more exhausting to participants than face-to-face meetings, the structure has to be adapted. Each discussion should not exceed 60 to 75 minutes and longer breaks of about 15 to 30 minutes in-between sessions are highly recommended. We have also replaced welcome rounds with icebreaker intros and made a list of all participants with their names and positions. Moderation and logging responsibilities should be separated as both tasks require substantial attention. The host of the meeting should be able to control access to meetings (e.g., via the waiting room functionality). Participants should also be reminded of the now common rules to only speak when requested as this ensures audio problems are kept to a minimum. Alternatively, questions can be posed via the chat function and then compiled by the moderator. Moreover, we have had good experiences with additional rounds for the expert group in-between the meetings with HEI members. These rounds do not replace attending a group dinner but they do make up for the absence of face-to-face contact. They also offer the opportunity to talk off the record. Lastly, the inspection of rooms could be replaced by videos sent in advance. Controversial matters could then also be inspected live with the help of smartphone cameras.

In recent months evalag has gained experience with virtual on-site visits (at universities, universities of applied sciences and arts academies), expert meetings and online seminars. Responses from both HEIs and experts have been mainly positive, with a small amount of critical feedback regarding personal contact perceived as missing. In conclusion, evalag is considering offering HEIs and experts virtual on-site visits as an alternative to face-to-face meetings in future as it could reduce travel costs and the ecological footprint of quality assurance activities. Another option is to mix both procedures with some experts attending the HEI in person and others joining the meetings online.