Mary Rozga, PhD, RDN, Emily Zitek, MS

Executive Summary

The Consumer Healthcare Experience State Survey (CHESS) is a state-level survey designed to capture consumer perspectives on healthcare affordability and system navigation. Developed by Altarum in 2018, CHESS addresses critical gaps in state-specific data by measuring both objective financial measures and subjective experiences of affordability, including the psychological burden and worry associated with healthcare costs.

CHESS employs a web-based panel methodology, surveying approximately 1,500 adults (ages 18+) in each participating state. The survey includes approximately 100 questions covering health insurance coverage and adequacy, health literacy, experiences with surprise medical bills and medical debt, system navigation, burden and worry related to affording care, healthcare shopping experiences, and preferred policy actions. Responses are weighted using post-stratification methods to align with state demographic profiles based on U.S. Census data.

Since its launch, CHESS has been implemented in 37 states plus the District of Columbia, with 64 survey waves completed. Applications span legislative action, government reports, legal proceedings, academic research, media coverage, policy implementation, and public campaigns across multiple states.

CHESS is designed as a rapid, cost-effective tool for identifying trends, comparing subgroups, tracking changes over time, and informing policy discussions. The survey’s consistent methodology, large sample sizes, rigorous quality controls, and weighting support its use as a reliable source of consumer perspective on state-level healthcare affordability.

Introduction

Healthcare affordability remains a critical concern for Americans, yet state‑level data on consumer experiences with healthcare costs and financial burden are limited. National surveys provide valuable insights into healthcare spending and insurance coverage, but they often lack the state‑specific granularity needed to inform local policy decisions. In addition, many existing surveys emphasize objective financial metrics, such as out‑of‑pocket costs and insurance premiums, while giving less attention to the experiential and psychological dimensions of affordability, including difficult care decisions and healthcare‑related worry.

The Consumer Healthcare Experience State Survey (CHESS) was developed by Altarum’s Health Research and Insights team in 2018 to address these gaps. CHESS captures both objective financial measures and subjective experiences of affordability. The survey covers healthcare access, healthcare-related burdens, experiences navigating the healthcare system, engagement in the healthcare system, and preferred policy actions.¹ This paper describes the development, implementation, and analytical approaches used for the CHESS (Exhibit 1). It also clarifies appropriate uses and limitations to support a transparent and responsible interpretation of findings.

This paper is intended for multiple audiences. Researchers may focus on the sections detailing sampling, weighting, and analytical standards. Policymakers and state partners may find the overview of survey design, appropriate uses, and limitations most relevant. Potential partners and funders may focus on the survey’s scope, consistency, and demonstrated real-world impact.

Exhibit 1. CHESS Methodology: From Development to Impact

Survey Development and Design

CHESS was developed through a systematic process designed to capture comprehensive information about consumer healthcare experiences and affordability challenges.

Development Process

CHESS was developed in 2018 by Altarum’s healthcare affordability and survey methodology experts through a comprehensive literature review of existing national and state-level surveys. This review identified gaps in available data and informed question selection to ensure coverage of both financial impacts and firsthand experiences.

The survey was pilot tested in Connecticut and Pennsylvania with 800 respondents in each state. This pilot phase informed key methodological decisions, including the target sample size of 1,500 respondents per state to support meaningful subgroup analyses, and the weighting approach to match state-specific demographic profiles. The survey continues to be reviewed and refined annually to reflect major concerns and changes in the healthcare landscape.

Survey Instrument

The CHESS questionnaire captures consumers’ experiences with healthcare affordability and system navigation. The survey includes questions on demographic and socioeconomic characteristics; health insurance coverage and adequacy; health literacy and understanding of healthcare costs; experiences with surprise medical bills and medical debt; interactions with and navigation of the healthcare system; burden and worry related to affording medical care or accessing needed services; and experiences shopping for healthcare. The survey also asks respondents about priorities for policy actions to address healthcare affordability and access challenges.

The survey includes approximately 100 questions, though the exact number varies by respondent based on branching logic. Question formats include Likert scales, ranking questions (where respondents select up to three priorities from a list), categorical response options, and limited open-ended questions. The median completion time is approximately 20 minutes. To support accessibility, the survey is offered in both English and Spanish.

Sampling Methodology

CHESS is designed to capture the experiences of adults residing in participating states using a web-based survey panel.

Target Population

The target population for CHESS consists of adults aged 18 and older who reside in the participating state.

Sample Design

CHESS employs a web-based panel approach through a partnership with an established online panel vendor.² This methodology was selected for its cost-effectiveness, speed of data collection, and ability to reach geographically dispersed populations while maintaining demographic targeting capabilities. The vendor maintains a large panel of individuals who have agreed to participate in survey research, and panel members are recruited through the vendor’s established processes. Additional discussion of the limitations and appropriate use of web-based panels is provided in the Limitations section below. Each state survey includes a demographically balanced sample of approximately 1,500 respondents.

Data Collection Procedures

Survey Administration

CHESS is administered online using the Lime Survey platform.³ The typical fielding period for each state is approximately three weeks. Panel members who have registered with the survey vendor have provided consent to participate in survey research as part of their enrollment process, satisfying informed consent requirements.

Privacy and Data Security

To protect respondent privacy and confidentiality, no personally identifiable information (such as names, addresses, or birthdates) is collected as part of the survey. All survey data are secured with VPN-required access to ensure data security and prevent unauthorized use.

Quality Control Measures

Multiple quality control procedures are implemented to ensure data integrity and response quality, including:

• Skip logic to guide respondents through appropriate question sequences;
• IP address tracking to prevent duplicate responses;
• Exclusion of respondents completing the survey in less than half of the median completion time; and
• Face validity checks comparing response patterns across states to identify anomalies.

Analytical Methodology

CHESS analyses follow standardized procedures to ensure results are consistent and interpretable across states and survey waves.

Data Preparation and Weighting

Following data collection, responses undergo standardized data cleaning procedures. Missing responses are retained in the dataset and treated as zero values for analysis; no case deletions or statistical imputations are performed. Survey data are weighted using post-stratification methods to match U.S. Census benchmarks for age, sex, race/ethnicity, and household income, supporting demographic alignment at the state level.

Reporting Standards and Thresholds

CHESS employs statistical thresholds to guide reporting and interpretation of results. Results are generally reported only when minimum thresholds are met, including a minimum sample size (n ≥ 100) and a coefficient of variation below 0.30. Estimates are reported with 95 percent confidence intervals. Results falling below these thresholds may be reported at the request of state partners, with clear caveats indicating that findings should be interpreted with caution.

Analysis Approaches

The primary analytical approach for CHESS is descriptive frequency analysis. For most survey questions, results are reported as percentages or frequencies. For ranking questions, where respondents select up to three priorities from a list of options, responses are aggregated by calculating how often each option appears among the selected responses, with all selections weighted equally.

Results may be analyzed at the state level overall or disaggregated by demographic and geographic subgroups through cross-tabulation. Common subgroup analyses include breakdowns by income level, age group, sex, race/ethnicity, insurance type, and geographic setting (urban/rural). Specific analyses are determined in collaboration with state partners based on organizational priorities and data limitations.

Open-ended survey responses are systematically coded using Excel and themed to enable analysis of qualitative data. All quantitative analyses are conducted using SAS 9.4 statistical software,⁴ with automated programming to ensure consistency and accuracy in calculations and output generation.

Aggregation Methodology for National Benchmarks

State level data are aggregated across applicable metrics to produce a national-level data set resulting in national benchmarks that support meaningful state and year-to-year comparisons with larger, more robust sample sizes for statistical analysis. By combining data across states, the resulting estimates provide a more reliable sample from which to make inferences about the broader population. This approach relies on standardization across survey waves, including maintaining question consistency and systematically handling any modifications. It also includes the identification of potential outlier states with significantly different results, which can be evaluated for possible exclusion from the national dataset. Aggregated data are weighted to reflect national population characteristics, with demographic balancing aligned to national averages for age, sex, race and ethnicity, income, and geography.

Validity and Reliability

Validity and reliability are assessed using multiple complementary approaches. Content, construct, and criterion validity are supported through careful survey design to ensure that items comprehensively capture the domains of interest, align with underlying conceptual frameworks, and demonstrate appropriate relationships with related external measures where available. Where applicable, individual metrics will be evaluated as predictors of broader measures to further support survey validation.

Reliability is evaluated through assessments of cross-state consistency and temporal stability, examining whether observed patterns are comparable across states and remain stable across survey waves. Together, these analyses provide evidence that the measures yield consistent and credible results suitable for national-level benchmarking and longitudinal comparisons.

Limitations and Mitigation Strategies

Like all survey-based research, CHESS has methodological limitations that are important to consider when interpreting results.

Web-Based Panel Limitations

CHESS employs a convenience sample drawn from an online panel rather than a probability-based sample, which limits generalizability to the full state population. The survey also excludes individuals without internet access and those who do not participate in online panels, potentially introducing selection bias. Research indicates that online panel members tend to have more education and higher socioeconomic status than the general population.^5,6

Despite these limitations, web-based panel surveys are an established and widely used methodology in health services and policy research.^5,7 Major public health and research organizations employ panel-based approaches to produce timely, policy-relevant data, including the CDC’s National Center for Health Statistics (NCHS) Rapid Surveys System and the Kaiser Family Foundation’s Health Tracking Poll.^8,9,10

Online panels offer important advantages including cost-effectiveness, rapid data collection, and the ability to reach geographically dispersed populations while supporting demographic targeting.^11,12 While they do not replace probability-based surveys, they are widely considered “fit for purpose” for tracking trends and informing policy discussions when paired with transparent reporting and safeguards such as quota sampling and post-stratification weighting.^11,13

Self-Report and Data Quality Considerations

CHESS relies on self-reported data, which may be subject to recall and social desirability bias. However, self-report is essential for capturing experiential and psychological dimensions of healthcare burden that cannot be measured using administrative data. Evidence from systematic reviews support the validity and reliability of patient-reported experience measures.^14,15,16

Emerging challenges for online surveys include the potential for automated, bot‑generated, or AI‑assisted responses. CHESS addresses this risk through multiple quality‑control safeguards, including completion‑time thresholds, duplicate prevention, and review of response patterns for internal consistency and face validity. Ongoing monitoring and refinement of quality‑control procedures remain an important component of maintaining data integrity in a rapidly evolving survey environment.

Coverage, Representation and Appropriate Use

The digital divide may result in disenfranchisement of some populations, including older adults, low-income individuals, and rural residents. CHESS addresses these concerns through demographic balancing during survey fielding, post-stratification weighting and transparent documentation of sample characteristics.

CHESS is designed to:

• • Identify trends and patterns in healthcare experiences;
• • Compare subgroups within the survey sample;
• • Track changes over time using consistent methodology; and
• • Inform policy discussions and decision making.

CHESS is not intended to produce precise population-level prevalence estimates. To reinforce appropriate interpretation, CHESS reports refer to “respondents” rather than “residents.”

Strengths of CHESS

CHESS offers several strengths that support its use as a durable, policy‑relevant data source for understanding healthcare affordability and consumer experience.

Methodological Strengths

CHESS is grounded in rigorous and transparent methodological approaches designed to support consistent measurement across states and over time. The survey was developed through a comprehensive literature review by healthcare affordability and survey methodology experts and has been refined annually since 2018 to reflect changes in the healthcare landscape.

Standardized survey design and large state-level sample sizes enable meaningful subgroup analyses and trend tracking. Quality control procedures, including completion time screening, duplicate response prevention, and face validity checks, support data integrity. Post-stratification weighting aligns survey samples with state demographic profiles using U.S. Census benchmarks, supporting responsible interpretation of findings.

Unique Value Proposition

CHESS fills a distinct gap in the healthcare data landscape by capturing dimensions of affordability that are not well measured in existing national or administrative datasets. While many surveys focus on spending levels or insurance coverage, CHESS centers on consumer-reported financial strain, trade-offs, and worry, providing insight into how healthcare costs are experienced and managed by consumers and their families.

CHESS provides state-level granularity that is uncommon in healthcare surveys, giving states practical data about their own residents’ experiences. The rapid turnaround from fielding to results enables timely insights into policy development and decision-making. CHESS works with state organizations to tailor analyses to state priorities and policy-relevant questions, maximizing the survey’s utility for advancing state-level healthcare policy goals. The cost-effective approach enables repeated measurement over time, allowing states to track trends, assess policy impacts, and monitor emerging affordability challenges. Together, these features position CHESS as both an analytical tool and an ongoing policy resource.

Impact and Contributions

Since its launch in 2018, CHESS has demonstrated sustained real-world utility across policy and research settings. CHESS findings have directly informed legislative and executive action,_^17,18,19 state government reports,²⁰ legal proceedings,²¹ and academic research.^22,23 Additional applications include media coverage,²⁴ policy implementation guidance,²⁵ and public engagement campaigns.²⁶ This breadth of use demonstrates the survey’s role in making healthcare affordability challenges both visible and actionable (Exhibit 2).

By providing reliable, state-specific data on consumer experiences, CHESS enables policymakers and supporters to move beyond anecdotal evidence to understand the scope and nature of affordability challenges facing their constituents. The consistency of findings across states and survey waves establishes healthcare affordability as a persistent, measurable policy challenge that demands systematic solutions. CHESS’s focus on consumer-reported burden and worry provides evidence that resonates across political divides, elevating healthcare affordability as a priority for state-level policy action.

Exhibit 2. CHESS Real-World Impact: Supporting Policy Development and Action

Conclusion

The Consumer Healthcare Experience State Survey provides a rigorous and policy-relevant approach to measuring healthcare affordability and consumer experiences at the state level. By combining standardized methods and a focus on firsthand experience, CHESS fills a critical gap in state health policy data infrastructure.

While the survey has inherent limitations related to web-based sampling and self-report, these are clearly documented and mitigated through appropriate safeguards. CHESS’s state-level focus, rapid turnaround, and emphasis on consumer-reported burden make it a uniquely valuable tool for policymakers, supporters, and researchers.

References

1. Healthcare Value Hub. (2025). Consumer Healthcare Experience State Survey. https://healthcarevaluehub.org/consumer-healthcare-experience-state-survey/
2. Dynata. (2025). Data and Surveys. https://www.dynata.com/
3. LimeSurvey. (2025). LimeSurvey. https://www.limesurvey.org/
4. SAS Institute Inc. SAS software. Preprint posted online 2023:9.4.
5. Hays RD, Liu H, Kapteyn A. (2015). Use of Internet Panels to Conduct Surveys. Behav Res Methods 47(3):685-690. doi:10.3758/s13428-015-0617-9
6. Mercer, A., Lau, A. (2023, September 7). Comparing Two Types of Online Survey Samples. Pew Research Center. https://www.pewresearch.org/methods/2023/09/07/comparing-two-types-of-online-survey-samples/
7. Cornesse C, Blom AG, Dutwin D, et al. (2020, January 9). A Review of Conceptual Approaches and Empirical Evidence on Probability and Nonprobability Sample Survey Research. Journal of Survey Statistics and Methodology 8(1); 4-36. https://doi.org/10.1093/jssam/smz041
8. National Center for Health Statistics. (2025, July 30). Survey Methods and Data Releases. Centers for Disease Control and Prevention. https://www.cdc.gov/nchs/rapid-surveys/about/survey-methods-data-releases.html
9. Blumberg, S. (2025). Establishing the NCHS Rapid Surveys System: Challenges and Lessons Learned. National Center for Health Statistics at the Centers for Disease Control and Prevention. https://surveydatascience.isr.umich.edu/wp-content/uploads/2024/09/Stephen-Blumberg-SLIDES-Rapid-Surveys-for-ISR-Jan-2025.pdf
10. Sparks, G., Montalvo, J., Montero, A., Mulugeta, M., Kirzinger, A., Lopes, L., and Hamel, L. (2025, October 24). KFF Health Tracking Poll: Public Use and Trust in Health Care Apps and Websites. Kaiser Family Foundation. https://www.kff.org/public-opinion/kff-health-tracking-poll-public-use-and-trust-in-health-care-apps-and-websites/
11. Freeland, E. (2024, November 1). Using Online Panels for Survey Research. https://psrc.princeton.edu/document/371
12. Lehdonvirta, V., Oksanen, A., Räsänen, P., Blank, G. (2020, April 29). Social Media, Web, and Panel Surveys: Using Non-Probability Samples in Social and Policy Research. Policy and Internet 13(1); 134-155. https://doi.org/10.1002/poi3.238
13. Baker, R., Brick, M., Bates, N., et. al. (2013, September 26). Summary Report of the AAPOR Task Force on Non-Probability Sampling. Journal of Survey Statistics and Methodology 1(2); 90-143. https://doi.org/10.1093/jssam/smt008
14. Anufriyeva, V., Pavlova, M., Stepurko, T., & Groot, W. (2021). The validity and reliability of self-reported satisfaction with healthcare as a measure of quality: a systematic literature review. International Journal for Quality in Health Care, 33(1). https://doi.org/10.1093/intqhc/mzaa152
15. Tzelepis, F., Sanson-Fisher, R. W., Zucca, A. C., & Fradgley, E. A. (2015). Measuring the quality of patient-centered care: why patient-reported measures are critical to reliable assessment. Patient preference and adherence, 9, 831–835. https://doi.org/10.2147/PPA.S81975
16. Alwan N. A. (2026). The Stigma of self-report in health research: Time to reconsider what counts as “Objective”. PLOS global public health, 6(1), e0005521. https://doi.org/10.1371/journal.pgph.0005521
17. Washington State House Committee on Health Care and Wellness. (2023). House Bill Report HB1508. Washington State Legislature. https://app.leg.wa.gov/committeeschedules/Home/Document/254755
18. Braun M. (2025). Indiana Executive Order 25-21. State of Indiana.
19. Gottfried, R. (2021). Bill No. A07363A. New York State Assembly. https://nyassembly.gov/leg/?default_fld=&leg_video=&bn=A07363&term=2021&Summary=Y&Actions=Y&Committee%26nbspVotes=Y&Floor%26nbspVotes=Y&Memo=Y&Text=Y
20. 2024 State Health Report. (2024). Washington State Board of Health. https://sboh.wa.gov/sites/default/files/2024-06/2024%20State%20Health%20Report%20V3.pdf
21. State of Michigan in the Supreme Court. (2024). Amici Curiae Brief on Behalf of Corewell Health and McLaren Health Care re: Baher and Jomaa v. Prime Healthcare Services – Garden City, Supreme Court No. 165377. https://www.courts.michigan.gov/494544/siteassets/case-documents/briefs/msc/2023-2024/165377/165377_101_ac_brf-corewell.pdf
22. Li, N., Panchal, R., Giannouchos, T., Pan, R. J., Nguyen, D., Nohavec, R., Britton, L., Chaiyakunapruk, N., Biskupiak, J., Wilson, A., & Brixner, D. (2024). The impact of a statewide insulin copay cap policy for insured patients with diabetes in Utah. Journal of managed care & specialty pharmacy, 30(2), 112–117. https://doi.org/10.18553/jmcp.2024.30.2.112
23. Fuse Brown, E. (2023, January 23). Private Sector Health Care Prices – Defining the Terms of the Policy Debate. Health Affairs Forefront. 10.1377/forefront.20230118.645515
24. Botkin B. (2024, March 13). Government Health Oregon Prescription Drug Affordability Board seeks public input about drug costs. Oregon Capital Chronicle. https://oregoncapitalchronicle.com/2024/03/13/oregon-prescription-drug-affordability-board-seeks-public-input-about-drug-costs/
25. Healthcare Benchmark Initiative Recommendations to the General Assembly. (2024, October 18). Connecticut Office of Health Strategy. https://portal.ct.gov/ohs/-/media/ohs/cost-growth-benchmark/reports-and-updates/ohs-healthcare-benchmarks-initiative-recommendations-2024.pdf
26. Prescription Drug Affordability Community Forums: Marylanders Tell Their Stories. (2022). Maryland Health Care for All. https://pdab.maryland.gov/documents/stakeholders/2022_Rx_affordability_forums_rpt_final.pdf