Accurate prediction of survey response rates is essential for optimizing survey design and ensuring high-quality data collection. Traditional methods often struggle to capture the complexity and multidimensionality of organizational datasets. This study applies the extreme Gradient Boosting (XGBoost) algorithm to predict response rates using organizational and demographic features. The model was trained on features including age, gender, job level, send hour, weekday, allowed response window, number of reminders, and total sent forms. The XGBoost model achieved strong predictive performance with an R2 score of 0.85 and a Mean Squared Error (MSE) of 0.02, reflecting the high accuracy in predicting response rates. Analysis of feature importance revealed that sent forms (46.6%) and Reminder (42.6%) were the most influential factors, while job_level (2.55%) and weekday (2.67%) also contributed to response behavior. Scatter plots of actual versus predicted response rates confirmed minimal deviation, demonstrating the reliability of the model. These results highlight the potential of machine learning techniques, particularly XGBoost, in accurately modeling survey response rates. Understanding feature importance allows researchers and organizations to strategically adjust survey design elements such as the number of invitations and reminders, to maximize participation.
References
[1]
Amirshahi, A., Kirsch, N., Reymond, J., & Baghersalimi, S. (2023). Predicting Survey Response with Quotation-Based Modeling: A Case Study on Favorability Towards the United States. 2023 10th IEEE Swiss Conference on Data Science (SDS) (pp. 1-8). IEEE. https://doi.org/10.1109/sds57534.2023.00008
[2]
Barkho, W., Carnes, N. C., Kolaja, C. A., Tu, X. M., Boparai, S. K., Castañeda, S. F. et al. (2024). Utilizing Machine Learning to Predict Participant Response to Follow-Up Health Surveys in the Millennium Cohort Study. Scientific Reports, 14, Article No. 25764. https://doi.org/10.1038/s41598-024-77563-8
[3]
Ben-David, E., Ibrahim, S., Mazumder, R., & Radchenko, P. (2021). Predicting Census Survey Response Rates via Additive Regression with Interactions. Annals of Applied Statistics, 19, 1-28.
[4]
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). ACM. https://doi.org/10.1145/2939672.2939785
[5]
Dinov, I. D. (2023). Model Performance Assessment, Validation, and Improvement. In editor (Ed.), The Springer Series in Applied Machine Learning (Vol. 2853, pp. 477-531). Springer International Publishing. https://doi.org/10.1007/978-3-031-17483-4_9
[6]
Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics, 29, 1189-1232. https://doi.org/10.1214/aos/1013203451
[7]
Groves, R. M., & Peytcheva, E. (2008). The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. Public Opinion Quarterly, 72, 167-189. https://doi.org/10.1093/poq/nfn011
[8]
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning. Springer.
[9]
Ibrahim, S., Mazumder, R., Radchenko, P., & Ben-David, E. (2021). Predicting Census Survey Response Rates with Interpretable Nonparametric Additive Modelsand Structured Interactions. arXiv:2108.11328v3. https://arxiv.org/abs/2108.11328v3
[10]
Kern, C., Weiss, B., & Kolb, J.-P. (2019). A Longitudinal Framework for Predicting Nonresponse in Panel Surveys. arXiv:1909.13361. https://arxiv.org/abs/1909.13361
[11]
Kern, M. et al. (2024). Calibration and XGBoost Reweighting to Reduce Coverage and Non-Response Biases in Overlapping Panel Surveys: Application to the Healthcare and Social Survey. BMC Medical Research Methodology, 24, Article No. 36. https://doi.org/10.1186/s12874-024-02171-z
[12]
Mazumder, R., Ben-David, E., & Ibrahim, S. (2021). Predicting Census Survey Response Rates via Interpretable Nonparametric Additive Models with Structured Interactions. arXiv:2108.11328v2. https://arxiv.org/pdf/2108.11328v2
[13]
Moeini, R., & Cultural Infusion Research Team (2023). Cultural Diversity Measurement through Diversity Atlas: A Case Study Approach. Cultural Infusion White Paper.
[14]
Moieni, R., & Mousaferiadis, P. (2022). Analysis of Cultural Diversity Concept in Different Countries Using Fractal Analysis. The International Journal of Organizational Diversity, 22, 43-62. https://search.proquest.com/openview/2e4c42e8af84f2c0a56d02dac5e0d983/1?pq-origsite=gscholar&cbl=5529398
[15]
Moeini, R., Mousaferiadis, P., & Pateel, P. (2022). An Analytical Approach to Measure the Cultural Diversity Mutuality between Two Communities. NeuroQuantology.
[16]
Xue, J. (2024). Optimization of Big Data Analysis Resources Supported by XGBoost and LSTM. Journal of Big Data Analytics, 3, 45-58.
[17]
Zhang, Y., & Zheng, Y. (2023). Estimating Response Propensities in Nonprobability Surveys Using Machine Learning. Journal of Survey Statistics and Methodology, 11, 123-145.
