More connection. More collection.
The California Health Survey (CHIS) is using a new multi-pronged data collection method — a series of communications by mail and the option to take the survey over the internet — to increase survey response.
Since its inception in 2001, the California Health Interview Survey (CHIS) has been an innovative and invaluable resource for policymakers, researchers, health experts, members of the media, and others for credible and comprehensive data on the health of Californians. For 20 years, the CHIS sample design and data collection methodology have remained relatively unchanged using random-digit dialing (RDD) computer-assisted telephone interviews (CATI), with the addition of cell phone RDD following the tremendous growth of cell phone usage.
Over the last few decades, the telephone survey landscape has seen some major changes as new barriers and technologies have developed. The growth of cell phone only households and advent of the smartphone resulted in large cultural shifts in how we communicate. Telephone interviewing has become more challenging with the proliferation of spam calls, robocalls, and call blocking, as well as the implementation of the Telephone Consumer Protection Act.
Simultaneously, new and more complete sampling frames based on addresses were developed and allowed for more survey modes to be easily used for data collection giving to the popular rise of mixed mode surveys. Accessibility to the Internet within homes and on our mobile devices also brought a new, less expensive way to collect data.
With all of these factors impacting survey research, the sustainability of a telephone-based methodology for CHIS was in question. It was time for the CHIS once again to innovate, and to explore alternative survey methods to remain relevant and viable.
Thanks to generous support from Kaiser Permanente and the California Department of Health Care Services, in 2018 CHIS was able to conduct two field experiments to test a new sample design and data collection methodology separately from our 2018 CHIS data collection. These experiments allowed us to evaluate different methods for web and telephone surveys, and prove the success of these new methods for the CHIS. The results of these innovative experiments have been presented at survey methodology conferences both here in the United States and abroad, and detailed reports are available below:
- Spring 2018 address-based sampling mail push-to-web field experiment
- Fall 2018 address-based sampling mail push-to-web statewide pilot experiment
Based on these successful experiments, beginning in 2019, CHIS transitioned to a mixed-mode survey (web and telephone) using a random sample of California addresses. After randomly selecting an address, CHIS will mail an invitation letter to the household inviting one adult member of the household to complete the survey online. This letter provides important information about the study including how to securely logon to the study website.
Following the initial invitation, additional mailings are sent to the household encouraging their participation. Once a household has received all letters, interviewers will attempt to complete the interview over the telephone for households where we have a valid telephone number. Households will also be provided a telephone number in the invitation letter that they can call if they prefer to complete the survey with a trained telephone interviewer, rather than completing the survey online.
Households that have eligible children (aged 0 to 11) will answer questions about one randomly selected child as part of the survey. Households that have eligible adolescents (aged 12 to 17) will be invited to have one randomly selected adolescent complete the survey online or by telephone, with a parent’s permission.
The online survey was initially available in five languages: English, Spanish, Chinese, Korean, and Vietnamese. A sixth language, Tagalog, was added later. Trained interviewers will be available to conduct the survey over the telephone in the same languages. The mailings sent to each address will be customized to include messages in multiple languages, based on the expected language needs in that neighborhood.
This new methodological approach allows CHIS to more effectively contact and survey Californians about their health and health care needs while maintaining the high level of data quality users have come to expect. This new design also reduces the number of households we need to contact each year while maintaining historic levels of completed surveys, thus reducing the burden on Californians each year. With these design changes in place, the redesigned CHIS is positioned to remain a valuable source of health data for years to come.