Immunizations have substantially reduced vaccine-preventable morbidity and mortality worldwide [1]. The Childhood Immunization Status (CIS) Combination 10 (Combo 10) measure is a core HEDIS (Healthcare Effectiveness Data and Information Set) pediatric quality indicator [2]. Combo 10 assesses whether children receive the complete series of 10 recommended vaccines by 24 months of age; however, national performance remains below desired benchmarks [3]. Identifying component-level drivers of Combo 10 failure is a public health informatics problem: it converts population-level quality data into actionable information for health plans, immunization programs, and policy stakeholders.

Traditional Combo 10 reporting summarizes overall completion but gives limited insight into component-level drivers. Public health informatics can extend routine quality reporting by linking population data, transparent analytics, and actionable interpretation. Explainable artificial intelligence (XAI) can show which inputs drive model output [4-9]. This study applied survey-weighted regression, random forest classification, and SHAP (Shapley Additive Explanations) to national survey data to identify components most predictive of HEDIS CIS Combo 10 completion.

The age-eligible cohort included 32,997 children; 16,021 had valid provider weights and complete outcome information. National HEDIS CIS Combo 10 completion declined from 53.7% in 2021 to 49.6% in 2022 and 44.6% in 2023, a 9.1-percentage-point decrease. The survey-weighted regression model was statistically significant for the annual trend (annual OR 0.83, 95% CI 0.831‐0.834; P<.001). Figure 1 shows nonuniform component declines, supporting component-level informatics analysis rather than aggregate rates alone.

Five-fold cross-validation was used to check model stability. Because CIS Combo 10 completion is defined by the 10 vaccine component indicators, classifier performance was expected to be near-perfect and was not interpreted as independent predictive validity. The random forest model was used for SHAP-based decomposition. SHAP rankings showed that influenza vaccination and rotavirus series completion had the largest mean absolute SHAP values and were therefore the strongest drivers of predicted Combo 10 status. DTaP and PCV had moderate importance, while MMR, varicella, IPV, Hib, and hepatitis B had lower SHAP influence. Figure 2 displays SHAP-based feature importance rather than native impurity-based importance measures.

The main finding was that incomplete influenza and rotavirus vaccination disproportionately explained HEDIS CIS Combo 10 underperformance in this national sample. This pattern is actionable because practices and health plans can contact patients who are due or overdue for immunizations, use visit-based prompts to reduce missed vaccines, and train staff to discuss immunization with families. Future motivational interviewing programs could pair these outreach workflows with family-centered communication and could be adapted to other HEDIS measures to improve public health quality.

This study illustrates how XAI can complement quality-measure reporting within public health informatics. Rather than treating machine learning as a black box, SHAP linked each prediction to vaccine-specific inputs and distinguished components merely included in HEDIS from those driving noncompletion. The approach can support transparent, data-driven quality improvement for clinicians, health plans, public health agencies, and policy stakeholders [5,6,8,9].

Limitations include use of public-use NIS-Child data rather than plan-level HEDIS claims and reliance on a global SHAP bar plot. SHAP values indicate model-based component contributions rather than causal effects; future public health informatics work should validate findings in plan-level HEDIS data and examine subgroup patterns with beeswarm or dependence plots.