An Explainable Machine Learning Framework for Thyroid Disease and Thyroid Cancer Prediction

Abstract

Thyroid disease is a global health issue, which requires the availability of accurate and interpretable diagnostic systems to assist in making early medical decisions. Recently, machine learning (ML) has demonstrated potential in predicting thyroid disease. However, it has not been fully solved due to the limitations of feature redundancy, model transparency, and clinical interpretability. This study addresses these challenges and proposes an interpretable ensemble learning system coupled with the Chi-square (χ²) feature selection method for effective prediction of thyroid disease and then uses the optimized features to train the Random Forest classifier. The proposed framework is validated on the benchmark datasets of thyroid diseases using various popular machine learning algorithms such as Random Forest, Gradient Boosting, AdaBoost, Decision Tree, K-Nearest Neighbors (KNN), Logistic Regression, Support Vector Machine (SVM), and Naive Bayes for the robust evaluation. Accuracy, precision, recall, F1-score, and ROC-AUC are the measurements of the correctness of the model. In addition, the explanation results are supplied by global and local explanations to make the prediction results easier to read and transparent clinically. The experimental results indicate that the suggested CSSOA- RF framework obtains better performance in some experimental circumstances. It achieves an accuracy, precision, recall, and F1-score of 99.87% on the benchmark dataset, which is higher than Random Forest (99.74%), Gradient Boosting (99.74%), AdaBoost (99.60%), Decision Tree (99.47%), KNN (93.91%), Logistic Regression (93.64%), SVM (92.85%), and Naive Bayes (20.26%). In another comparison test, the proposed framework achieved a high accuracy (99.73%), precision (97.79%), recall (98.97%), and F1 score (98.33%), which further confirmed the strength and generalization capacity of the proposed framework in various experimental settings. The explainability analysis is always consistent and reveals the most significant predictors of thyroid disease, thyroid-stimulating hormone (TSH), total thyroxine (TT₄), and free thyroxine index (FTI), that is, thyroid parameters, and confirms their clinical relevance. The proposed CS-SOA-RF framework is an accurate, interpretable, and low-computation clinical decision support system that healthcare practitioners can use for the early detection and diagnosis of thyroid disorders.