Nested Ensemble Stacking Framework for Diabetes Prediction Using Diverse Machine Learning Models

Abstract

Medical data science confronts unique challenges when tasked with predicting specific symptoms of a disease that is particularly heterogeneous. This challenge is exemplified in predicting cases of diabetes. In this paper, we present a novel type of ensemble model that improves predictive performance on diverse datasets through the use of combination strategies. The base of our model consists of ten individual classifiers: Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), Decision Tree (DT), Random Forest (RF), Gradient Boosting (GB), Extreme Gradient Boosting (XGBoost), AdaBoost, Hard Voting Ensemble, and Soft Voting Ensemble. The ten base classifiers leverage different bias-variance trade-offs, and therefore, work complementary in different decision boundary configurations. In the meta-level, the prediction of the ten base models is stacked using a meta-classifier. Our model designs a nested structure to control variance and bias while leveraging the strengths of individual learners. Evaluation of the proposed model on publicly available diabetes datasets indicated improvements in accuracy, precision, recall, and F1-score when compared to standalone models and traditional ensembles. Our model is shown to leverage stacking based-nested ensembles in predicting early cases of diabetes, and as a result, is a step forward in the development of reliable clinical support systems.