A model for big data analytics to improve service delivery in South African public hospitals.

Abstract

This study addresses the challenge of improving service delivery in South African public hospitals by utilising Big Data analytics and machine learning techniques. It employs a quantitative approach within the CRISP-DM framework. The research objectives included identifying factors that influence Big Data analytics, assessing various analysis methods, identifying patterns to improve services, developing a predictive model, and evaluating its efficiency and effectiveness. Utilising a synthetic dataset (Final_healthcare_big_data.csv), no real hospital/electronic health record data were used. The study implemented feature engineering and employed a Stacking Classifier consisting of XGBoost, Random Forest, CatBoost, and LightGBM to build the final predictive multiclass healthcare model categorizing outcomes as 'Improved,' 'Unchanged,' or 'Worsened.' Initially, during the trial stage, various algorithms were trained, including SVM and logistic regression; however, they were later dropped while building the final model as they were not suitable for this study, due to extended training durations unsuitable for multiclass. SHAP analysis highlighted critical features such as Satisfaction_Rating, Age, and Log_Wait_Time, and the stacked classifier model achieved an accuracy of 84% with AUC values of 0.93, 0.95, and 0.95, respectively. Patterns, like Wait_Length_Interaction, which had an impact on worsened outcomes, were identified, and a Streamlit application was developed to provide real-time insights. The findings demonstrate the model's potential to enhance healthcare decision-making by providing a scalable and interpretable solution. This study advances data science by laying a strong foundation for healthcare analytics in South Africa, with implications for resource allocation and improving patient care efficiency.