Explainable artificial intelligence for multi-class delivery status prediction

Late deliveries reduce service quality and increase exception-handling costs in supply chains, while complex predictive models are often difficult to translate into managerial decisions. This study uses the 2025 SynDelay synthetic dataset comprising 155,488 observations. After removing identifier variables and variables available only after order receipt, three algorithms: Logistic Regression, Random Forest, and XGBoost, are evaluated on a stratified 20% test set. The results show that XGBoost achieves the highest Macro-F1 score of 0.4825, with a precision of 0.8753 and a recall of 0.5152 for the delayed-delivery class. SHAP analysis identifies shipping mode as the most influential predictor, followed by order value and geographic features. The findings illustrate the potential of combining prediction and explainability to support shipment-risk screening, while also indicating that predictive performance should be interpreted cautiously because of the synthetic nature of the data and class imbalance.