Abstract

From 2005 onwards, web data collection is used as the most common way for data collection within Statistics Netherlands. After a series of experiments and parallel data collection with other modes as CAPI and CATI, the CAWI approach became common practice. For the LFS, web data collection was implemented in 2012. In the absence of an interviewer, the advance letters are of great importance. This paper describes how the advance letters of Statistics Netherlands are composed, tested and how the view on advance letters and testing has changed in the past 10 years.
In the absence of an interviewer, the only way to invite persons or households to participate in surveys is via traditional mail. There is no register of e-mail addresses available at Statistics Netherlands. With mail being the only way to reach out to potential respondents in a non-CAPI/CATI design, this letter is the only way of communicating and thus is of great importance (e.g. De Leeuw, Callegaro, Hox, Korendijk, & Lensvelt-Mulders, 2007; Dillman, 2000; Goldstein & Jennings, 2002; Luiten 2011; Yammarino, Skinner & Childers, 1991).

Where CAWI as a data collection method was extensively tested before it was implemented, the advance letter design received much lesser attention. The old letter was long (about 1,5 page) and used quite complicated language. The length of the letter was the effect of all of the extensive information that was included. It used a not very tempting introduction into the survey and the explicit request to log in onto the questionnaire. After the introduction, there was some explanation about the sample and how we reached out to the specific person. Next, the web address, with login code and credentials were presented in the text and the rest of the letter contained information about privacy and a statement that the survey would be linked with other administrative records (informed consent).

In 2013, an external communication bureau was asked to redesign the letter, together with methodologists and experts from the statistical department. Without extensively testing, the letter was put into the field with disappointing results. The response rate dropped from 30% to 20% (Luiten and de Groot, 2014).

Further examination by qualitative interviewing was undertaken to understand these results. In this qualitative study the majority of the participants preferred the new letter and 45% said they would participate against 33% with the old letter. With the qualitative analyses pointing in in the opposite direction, it was clear that there was need for a different approach if we wanted to improve the advance letters. This paper describes the systematical approach that was taken. In an extensive series of experiments each aspect of the advance letter was studied and then adjusted by a team of experts and quantitatively tested. In the remainder of this paper we describe a number of the experiments we performed.