CALIBRATION-AWARE EVALUATION OF MACHINE LEARNING MODELS FOR SURVEY-BASED PREDICTION

Abstract

We evaluated whether school-property and electronic bullying indicators improve prediction of persistent sadness or hopelessness beyond a demographic baseline in the National Youth Risk Behavior Survey (YRBS). Baseline and expanded histogram gradient boosting models shared preprocessing and hyperparameters; evaluation combined discrimination, Brier score, calibration diagnostics, and paired uncertainty. Selection was retrospective, so the 2023 held-out partition supports internal comparison rather than an untouched final test. The selected Platt-calibrated expanded model achieved internal ROC AUC 0.716, average precision 0.644, and Brier score 0.206. Compared with the matched baseline, improvements were 0.066, 0.106, and 0.019, respectively, with conditional 95% intervals above zero. Applying the frozen pipelines to the independent 2021 wave preserved the incremental pattern, with expanded-model ROC AUC 0.716 and Brier score 0.205. Weighting sensitivities supported the comparison, while subgroup calibration varied. The findings illustrate calibration-aware feature-set evaluation and remain non-causal and research-only; they do not establish fairness or readiness for operational use.