Evaluation dashboard

The honest result is the useful result.

The champion ensemble performed well on low-risk cases but struggled with the transitional mid-risk class. These results come from a profile-isolated holdout.

Locked holdout · 93 rows

Accuracy 74.2% 69 of 93 predictions correct

Macro F1 0.600 Equal weight across risk classes

Balanced accuracy 0.596 Mean class recall

Quadratic kappa 0.639 Ordinal agreement beyond chance

Holdout accuracy by model

The diverse ensemble was selected using development data before holdout evaluation.

Ensemble74.2%

Extra Trees73.1%

Random Forest73.1%

CatBoost73.1%

XGBoost71.0%

LightGBM71.0%

SVC68.8%

Champion confusion matrix

Rows are observed classes; columns are predicted classes.

Observed risk

LowMidHigh Low5496.4%00.0%23.6% Mid1473.7%315.8%210.5% High422.2%211.1%1266.7%

Predicted risk

Low recall96.4%

Mid recall15.8%

High recall66.7%

Two benchmarks

Why the lower score carries more evidence.

The project retained a legacy benchmark for sensitivity analysis, then separated it from the leakage-resistant research result.

Profile-isolated evaluation 74.2%

Exact duplicates removed; identical predictor profiles kept in one split or fold. This is the reported research estimate.

versus

Legacy row split 90.1%

79.8% of test rows had a predictor profile also present in training. Useful as an audit, not as clinical or external validation.

Model-agnostic SHAP importance

Mean absolute change in predicted probability, aggregated over the explanation sample.

Systolic BP0.083

Blood glucose0.078

Body temp0.052

Diastolic BP0.020

Heart rate0.014

Age0.014

Permutation importance

Average decrease in holdout accuracy after shuffling each raw input.

Blood glucose0.137

Body temp0.070

Systolic BP0.062

Diastolic BP0.034

Age0.015

Heart rate0.006

Complete model comparison

No candidate dominated every criterion. Random Forest led macro F1; Extra Trees led balanced accuracy and high-risk recall.

Model	Accuracy	Balanced accuracy	Macro F1	QWK	Mid recall	High recall
Diverse ensemble	74.19%	0.5963	0.5999	0.6387	0.1579	0.6667
Extra Trees	73.12%	0.6145	0.6014	0.6157	0.2105	0.7222
Random Forest	73.12%	0.6135	0.6129	0.6231	0.2632	0.6667
SVC	68.82%	0.5655	0.5554	0.5531	0.2105	0.6111
LightGBM	70.97%	0.5775	0.5884	0.6042	0.2105	0.6111
XGBoost	70.97%	0.5659	0.5721	0.5798	0.1579	0.6111
CatBoost	73.12%	0.5797	0.5610	0.6067	0.0526	0.7222

Interpretation boundary

These results establish an internal, reproducible proof of concept. They do not establish safety, clinical utility, external validity, or performance for child and neonatal outcomes.