Ana Villar and Elena Sommer, ESS ERIC
The Cross-National Online Survey (CRONOS) panel designed and implemented as part of the SERISS project is the first attempt to establish a probability-based cross-national online panel following an input harmonisation approach. With fieldwork for the panel nearing completion, and the first data due to be released in early 2018, we provide an update on how the panel was implemented and some of the opportunities for analysis it will provide.
The purpose of CRONOS
Online panels offer opportunities for quick, flexible and cost-efficient data collection. There are, however, specific challenges associated with online panels. Probability-based online panels of the general population usually need to be recruited using an offline mode (e.g. face-to-face interview or postal invitation) because email lists from which a random probability sample could be drawn are rarely available. In the case of cross-national probability-based panels, the harmonised recruitment and centralised panel management are further challenged by different internet penetration rates across countries and legal issues related to cross-border data transfer.
CRONOS is a pilot study to evaluate the effectiveness of panel recruitment off the back of an existing cross-national face-to-face survey in terms of costs, sample representativeness, participation rates, panel attrition over time and data quality. It serves as a ‘proof of concept’ from which to produce a blueprint for an online probability-based panel that a range of cross-sectional survey infrastructures could consider adopting in the future.
CRONOS recruitment was done off the back of the European Social Survey (ESS) Round 8 (2016). After completing the ESS face-to-face interview, respondents in Great Britain, Estonia and Slovenia who were 18 or older were invited to participate in six 20 minutes online surveys over a period of twelve months. They were offered a £5/€5 incentive with each survey invitation as a token of appreciation for their participation. Respondents who did not have internet access for private use were offered a tablet and internet provision for the duration of the project to enable their participation.
Following a harmonised approach, the CRONOS panel implemented a centralised panel and administration system. The system was used to centrally monitor the recruitment and participation, and to send out email invitations and reminders. The surveys were centrally programmed and administered using Questback-EFS.
Around six in ten respondents to ESS Round 8 respondents agreed to join the CRONOS panel (”yes” or “maybe”): 59% in Estonia, 64% in Slovenia, and 67% in Great Britain. Actual participation was, as expected, lower than the initial agreement but was comparable with other online probability-based surveys: of panellists who received a web survey invitation, 76% in Estonia, 68% in Slovenia, and 56% in Great Britain completed the Wave 1 survey. Participation remained relatively stable over the four waves that have been completed so far, resulting in about 600 respondents in each participating country. An important next step will be to look further at these respondents in terms of sample representativeness.
The data collection started in December 2016 with a short 10-minute Welcome survey. The Wave 1 survey was sent out in February 2017. The last wave, Wave 6, will be conducted in January and February 2018. The surveys contain questions on diverse topics from SERISS partners – European Value Study (EVS), Generations and Gender Programme (GGP), European Social Survey (ESS) as well as other cross-national surveys including European Quality of Life Survey (EQLS), and the International Social Survey Programme (ISSP).
The CRONOS panel was, for example, used for pre-testing of the items that were developed as part of the questionnaire design process for the ESS Round 9 rotating modules and new items for the EVS 2017 and GGP 2020 surveys. CRONOS Wave 3 also contains country-specific benchmark items such as census questions and questions from large scale surveys like the European Union Statistics on Income and Living Conditions (EU-SILC) and the Survey of Adult Skills (PIAAC). Further, the CRONOS panel has served as a platform for experimental testing. Apart from contact mode and incentives experiments aimed at improving participation rates, motivational message experiments aimed at improving response quality, as well as various question wording and MTMM experiments, were implemented in several CRONOS waves.
The data collected online will be part of a larger dataset containing data from the face-to-face ESS Round 8 interview, allowing for new analyses that were not previously possible given ESS restrictions for interview time. Data from which respondents are not directly identifiable will be made publicly and freely available for non-commercial use.
The release of the survey data for Wave 0 – Wave 3 is planned for January 2018. The data and documentation will be freely available on the ESS website. Several deliverables describing the design and implementation of the CRONOS panel will be made available on the SERISS website starting from December 2017. The survey data for the remaining CRONOS waves (Wave 4 – Wave 6), recruitment data, as well as contact data and paradata for each wave will follow in the second half of 2018.