Evaluation of Tree-Based Ensemble Models Using Socioeconomic Data
DOI:
https://doi.org/10.21512/comtech.v17i2.14076Keywords:
air quality, socioeconomic, ensemble learning, bagging, boostingAbstract
Air quality significantly impacts public well-being and environmental sustainability, with socioeconomic factors known to influence air quality levels. However, existing AQI prediction studies predominantly rely on meteorological or pollutant variables, while the predictive capability of socioeconomic indicators remains underexplored. This study evaluates the effectiveness of tree-based ensemble learning techniques, including bagging, boosting, stacking, and multi-layer stacking, to predict air quality index (AQI) in Singapore using monthly socioeconomic indicators (34 variables) collected in Singapore from April 2014 until July 2024, encompassing transportation, tourism, industrialization, and price-related variables. Various tree-based algorithms, including Random Forest, ExtraTrees, XGBoost, and LightGBM, were evaluated. Stacking and multi-layer stacking models were also implemented with Linear Regression as the meta-learner. Feature engineering guided by the Pearson correlation coefficient was employed to optimize input selection. To optimize performance, hyperparameter tuning and model architectures were explored. The stacking model achieved the best predictive performance, with MAE, RMSE, and MAPE values of 13.68, 22.73, and 0.1359, respectively. Bagging models outperformed boosting models on the small dataset, while multi-layer stacking added computational complexity without significant performance gains. The results indicate that stacking captures complementary predictive patterns in socioeconomic data more effectively than deeper ensemble architectures, which offer limited benefits under data-constrained conditions. Furthermore, the findings show that socioeconomic variables provide sufficient predictive information to support AQI forecasting. Overall, this study highlights the potential of integrating socioeconomic data and offers methodological insights for future air quality research and applications.
References
Abayomi-Alli, O. O., Damaševičius, R., Abayomi-Alli, O. O., Damaševičius, R., Maskeliūnas, R., & Misra, S. (2022). An Ensemble Learning Model for COVID-19 Detection from Blood Test Samples. Sensors, 22(6), 2224. https://doi.org/10.3390/s22062224
Alazba, A., & Aljamaan, H. (2022). Software Defect Prediction Using Stacking Generalization of Optimized Tree-Based Ensembles. Applied Sciences, 12(9), 4577. https://doi.org/10.3390/app12094577
Allgaier, J., & Pryss, R. (2024). Cross-Validation Visualized: A Narrative Guide to Advanced Methods. Machine Learning and Knowledge Extraction, 6(2), 1378–1388. https://doi.org/10.3390/make6020065
Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785–794.
Department of Statistics Singapore. (2025). Singstat Website. Retrieved August 8, 2025, from https://www.singstat.gov.sg/
Gafarov, F., Berdnikov, A., & Ustin, P. (2022). Online social network user performance prediction by graph neural networks. International Journal of Advances in Intelligent Informatics, 8(3), 285. https://doi.org/10.26555/ijain.v8i3.859
Ghiasi, M. M., & Zendehboudi, S. (2021). Application of decision tree-based ensemble learning in the classification of breast cancer. Computers in Biology and Medicine, 128, 104089. https://doi.org/10.1016/j.compbiomed.2020.104089
Gurumoorthy, S., Kokku, A. K., Falkowski-Gilski, P., & Divakarachari, P. B. (2023). Effective Air Quality Prediction Using Reinforced Swarm Optimization and Bi-Directional Gated Recurrent Unit. Sustainability, 15(14), 11454. https://doi.org/10.3390/su151411454
Hu, L., & Li, L. (2022). Using Tree-Based Machine Learning for Health Studies: Literature Review and Case Series. International Journal of Environmental Research and Public Health, 19(23), 16080. https://doi.org/10.3390/ijerph192316080
Issah, I., Appiah, O., Appiahene, P., & Inusah, F. (2023). A systematic review of the literature on machine learning application of determining the attributes influencing academic performance. Decision Analytics Journal, 7, 100204. https://doi.org/10.1016/j.dajour.2023.100204
Jiao, K., Xu, M., & Liu, M. (2018). Health status and air pollution related socioeconomic concerns in urban China. International Journal for Equity in Health, 17(1), 18. https://doi.org/10.1186/s12939-018-0719-y
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: a highly efficient gradient boosting decision tree. Proceedings of the 31st International Conference on Neural Information Processing Systems, 3149–3157.
Koopialipoor, M., Asteris, P. G., Salih Mohammed, A., Alexakis, D. E., Mamou, A., & Armaghani, D. J. (2022). Introducing stacking machine learning approaches for the prediction of rock deformation. Transportation Geotechnics, 34, 100756. https://doi.org/10.1016/j.trgeo.2022.100756
Kumar, K., & Pande, B. P. (2023). Air pollution prediction with machine learning: a case study of Indian cities. International Journal of Environmental Science and Technology, 20(5), 5333–5348. https://doi.org/10.1007/s13762-022-04241-5
Kumar, M., Singhal, S., Shekhar, S., Sharma, B., & Srivastava, G. (2022). Optimized Stacking Ensemble Learning Model for Breast Cancer Detection and Classification Using Machine Learning. Sustainability, 14(21), 13998. https://doi.org/10.3390/su142113998
Lin, S.-Y., Kung, Y.-C., & Leu, F.-Y. (2022). Predictive intelligence in harmful news identification by BERT-based ensemble learning model with text sentiment analysis. Information Processing & Management, 59(2), 102872. https://doi.org/10.1016/j.ipm.2022.102872
Manisalidis, I., Stavropoulou, E., Stavropoulos, A., & Bezirtzoglou, E. (2020). Environmental and Health Impacts of Air Pollution: A Review. Frontiers in Public Health, 8. https://doi.org/10.3389/fpubh.2020.00014
Mao, W., Wang, W., Jiao, L., Zhao, S., & Liu, A. (2021). Modeling air quality prediction using a deep learning approach: Method optimization and evaluation. Sustainable Cities and Society, 65, 102567. https://doi.org/10.1016/j.scs.2020.102567
Méndez, M., Merayo, M. G., & Núñez, M. (2023). Machine learning algorithms to forecast air quality: a survey. Artificial Intelligence Review, 56(9), 10031–10066. https://doi.org/10.1007/s10462-023-10424-4
Menshawy, L., Eid, A. H., & Abdel-Kader, R. F. (2023). Ensemble deep models for COVID-19 pandemic classification using chest X-ray images via different fusion techniques. International Journal of Advances in Intelligent Informatics, 9(1), 51. https://doi.org/10.26555/ijain.v9i1.922
Mienye, I. D., & Sun, Y. (2022). A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects. IEEE Access, 10, 99129–99149. https://doi.org/10.1109/ACCESS.2022.3207287
National Environment Agency. (2025). Singapore’s open data portal. Retrieved August 8, 2025, from https://data.gov.sg/datasets?agencies=National+Environment+Agency+(NEA)&resultId=d_e1058d6974c877257e32048ab128ad83&page=1
Putri, N. S. F., Wibawa, A. P., Ar Rasyid, H., Nafalski, A., & Hasyim, U. R. (2023). Boosting and bagging classification for computer science journal. International Journal of Advances in Intelligent Informatics, 9(1), 27. https://doi.org/10.26555/ijain.v9i1.985
Rashid, M., Kamruzzaman, J., Imam, T., Wibowo, S., & Gordon, S. (2022). A tree-based stacking ensemble technique with feature selection for network intrusion detection. Applied Intelligence, 52(9), 9768–9781. https://doi.org/10.1007/s10489-021-02968-1
Ravindiran, G., Hayder, G., Kanagarathinam, K., Alagumalai, A., & Sonne, C. (2023). Air quality prediction by machine learning models: A predictive study on the indian coastal city of Visakhapatnam. Chemosphere, 338, 139518. https://doi.org/10.1016/j.chemosphere.2023.139518
Saied, M., Guirguis, S., & Madbouly, M. (2023). A comparative analysis of using ensemble trees for botnet detection and classification in IoT. Scientific Reports, 13(1), 21632. https://doi.org/10.1038/s41598-023-48681-6
Shafieian, S., & Zulkernine, M. (2023). Multi-layer stacking ensemble learners for low footprint network intrusion detection. Complex & Intelligent Systems, 9(4), 3787–3799. https://doi.org/10.1007/s40747-022-00809-3
Sharma, A., Priya, T. H., & Naidu, V. (2024). Optimizing bearing health condition monitoring: exploring correlation feature selection algorithm. Engineering Research Express, 6(2), 025511. https://doi.org/10.1088/2631-8695/ad3380
Singh, P., Pal, G. K., & Gangwar, S. (2022). Prediction of Cardiovascular Disease Using Feature Selection Techniques. International Journal of Computer Theory and Engineering, 14(3), 97–103. https://doi.org/10.7763/IJCTE.2022.V14.1316
Van, N. H., Van Thanh, P., Tran, D. N., & Tran, D.-T. (2023). A new model of air quality prediction using lightweight machine learning. International Journal of Environmental Science and Technology, 20(3), 2983–2994. https://doi.org/10.1007/s13762-022-04185-w
Wang, Y., Wang, Y., Xu, H., Zhao, Y., & Marshall, J. D. (2022). Ambient Air Pollution and Socioeconomic Status in China. Environmental Health Perspectives, 130(6). https://doi.org/10.1289/EHP9872
Yoon, H. I., Lee, H., Yang, J.-S., Choi, J.-H., Jung, D.-H., Park, Y. J., Park, J.-E., Kim, S. M., & Park, S. H. (2023). Predicting Models for Plant Metabolites Based on PLSR, AdaBoost, XGBoost, and LightGBM Algorithms Using Hyperspectral Imaging of Brassica juncea. Agriculture, 13(8), 1477. https://doi.org/10.3390/agriculture13081477
Zhang, Y., Liu, J., & Shen, W. (2022). A Review of Ensemble Learning Algorithms Used in Remote Sensing Applications. Applied Sciences, 12(17), 8654. https://doi.org/10.3390/app12178654
Zhiyong, L., Hongdong, Z., Ruili, Z., Kewen, X., Qiang, G., & Yuhai, L. (2019). Fault Identification Method of Diesel Engine in Light of Pearson Correlation Coefficient Diagram and Orthogonal Vibration Signals. Mathematical Problems in Engineering, 2019(1). https://doi.org/10.1155/2019/2837580
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Riechie, Ferry Vincenttius Ferdinand, Feliks Parningotan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License - Share Alike that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
 USER RIGHTS
 All articles published Open Access will be immediately and permanently free for everyone to read and download. We are continuously working with our author communities to select the best choice of license options, currently being defined for this journal as follows:
