Temporal Robustness and Lightweight Adaptation for Just-In-Time Software Defect Prediction

Abstract

Just-In-Time Software Defect Prediction (JIT-SDP) is commonly evaluated using historical software changes, yet deployment conditions are inherently temporal. This study investigates temporal robustness and lightweight within-project adaptation using a leakage-resistant chronological protocol. The original Brackets case study contains 11,601 changes and 14 change-level metrics; a strict 70/30 split yields 8,120 historical and 3,481 future observations. Recursive Feature Elimination (RFE), fitted only within historical development data, selects six features. The same protocol is then extended to four additional processed JIT-SDP streams from Tomcat, JGroups, Spring-Integration, and Camel. On Brackets, the frozen Random Forest obtains F1=0.5366, ROC-AUC=0.7961, and PR-AUC=0.5161. On the four additional projects, frozen F1 ranges from 0.2975 to 0.6741, demonstrating substantial project-level variation. Temporal segmentation shows marked future-period degradation in three of the four additional projects. Cumulative and fixed recency windows, together with a 4,000-record recency model using adaptive threshold selection, are compared without using future-test labels for model selection. Across the 16 project-period blocks, the Friedman test detects differences among strategies (p < 0.0001, Kendall’s W = 0.3271), but pairwise results do not establish a uniformly superior adaptation strategy; the 2,000-record window is significantly different from the frozen strategy (p = 0.0042) in the tested direction of lower mean F1. A published-baseline-style comparator produces mixed project-level differences and is explicitly distinguished from an exact reproduction of BORB. Bootstrap confidence intervals further show overlapping uncertainty between frozen and adaptive-threshold configurations for each project. The evidence supports temporal degradation and project-dependent adaptation behavior, rather than a claim of universal adaptation superiority.