Abstract

Financial institutions face significant credit risk when evaluating credit applications, making fraud detection and prevention a primary task for all financial institutions and service companies. After decades of research and development, the credit scoring model has been improved by artificial intelligence and machine learning. This research proposes a new multi-stage hybrid ensemble classification model for improving the efficiency of credit scoring applications by leveraging artificial intelligence and machine learning advancements. A new weighted ensemble filter and an improved borderline SMOTE for minority oversampling are proposed to enhance the dataset quality by identifying the most suitable set of features for analysis, addressing the bias caused by an imbalanced class distribution, and improving predictive accuracy. Moreover, to reduce the variance and enhance the accuracy, a new nested ensemble classification model was introduced to enhance the predictive accuracy of credit scoring performance. The proposed model outperformed four credit scoring datasets, achieving an improved accuracy and AuC of 92.43% and 0.968 for the Australian credit dataset, 83.16% and 0.901 for the German credit dataset, and an improved AUC of 0.938 for the Japanese credit dataset. The experimental findings validate the practical efficiency of the proposed model, which outperforms competing models owing to its unique combination of feature engineering, balancing techniques, and use of a novel nested ensemble classifier. The study offers financial institutions not only a more robust tool for credit scoring to make more reliable credit decisions but also reduces operational costs associated with manual evaluation processes, risk management, and fraud mitigation.

Keywords

Credit Scoring, Hybrid Ensemble Classification, Feature Importance, Feature Selection, Minority Oversampling,

Downloads

Download data is not yet available.

References

  1. W. Zhang, J. Wang, Credit risk contagion in complex companies network–Empirical research based on listed agricultural companies, Economic Analysis and Policy, 82, (2024) 938-953. https://doi.org/10.1016/j.eap.2024.04.025
  2. W. Zhang, D. Yang, S. Zhang, J.H. Ablanedo-Rosas, X. Wu, Y. Lou, A novel multi-stage ensemble model with enhanced outlier adaptation for credit scoring, Expert Systems with Applications, 165, (2021) 113872. https://doi.org/10.1016/j.eswa.2020.113872
  3. W.A. Addy, A.O. Ajayi-Nifise, B.G. Bello, S.T. Tula, O. Odeyemi, T. Falaiye, AI in credit scoring: A comprehensive review of models and predictive analytics, Global Journal of Engineering and Technology Advances, 18(2), (2024) 118-129. https://doi.org/10.30574/gjeta.2024.18.2.0029
  4. M. Naved, R. Kumar, S.S Saheb, Analyzing financial stability by predicting bankruptcy situations with machine learning, Journal of Artificial Intelligence and System Modelling, 1(3), (2024) 18-35. https://doi.org/10.22034/jaism.2024.457068.1039
  5. S. Abimannan, E.S.M. El-Alfy, Y.S. Chang, S. Hussain, S. Shukla, D. Satheesh, Ensemble multifeatured deep learning models and applications: A survey IEEE Access, 11, (2023) 107194-107217. https://doi.org/10.1109/ACCESS.2023.3320042
  6. M. Mahbobi, S. Kimiagari, M. Vasudevan, Credit risk classification: an integrated predictive accuracy algorithm using artificial and deep neural networks, Annals of Operations Research, 330(1), (2023) 609-637. https://doi.org/10.1007/s10479-021-04114-z
  7. Ghavidel, P. Pazos, Machine learning (ML) techniques to predict breast cancer in imbalanced datasets: a systematic review, Journal of Cancer Survivorship, 19(1), (2025) 270-294. https://doi.org/10.1007/s11764-023-01465-3
  8. S.K. Rath, M. Sahu, S.P. Das, J.J. Jena, C. Jena, B. Khan, A. Ali, P. Bokoro, Software reliability prediction using ensemble learning on selected features in imbalanced and balanced datasets: A review, Computer Systems Science and Engineering, 48(6), (2024) 1513-1536. https://doi.org/10.32604/csse.2024.057067
  9. S. Farhadpour, T.A. Warner, A.E. Maxwell, Selecting and interpreting multiclass loss and accuracy assessment metrics for classifications with class imbalance: Guidance and best practices, Remote Sensing, 16(3), (2024) 533. https://doi.org/10.3390/rs16030533
  10. P. Ramila Rajaleximi, M.S. Irfan Ahmed, A. Alenezi, Classification of imbalanced class distribution using random forest with multiple weight based majority voting for credit scoring, International Journal of Recent Technology and Engineering, 7(6S5), (2019) 517-526. https://www.ijrte.org/wp-content/uploads/papers/v7i6s5/F10910476S519.pdf
  11. K. Hemapriya, K. Valarmathi, Innovative framework for thyroid disease detection by leveraging hybrid AGTEO feature selection and GRU classification model, International Research Journal of Multidisciplinary Technovation, 6(3), (2024) 112-127. https://doi.org/10.54392/irjmt2439
  12. P. Kumar, U.L. Maneesh, G.M. Sanjay, Optimizing Loan Approval Decisions: Harnessing Ensemble Learning for Credit Scoring, In Proc. 2024 International Conference on Advances in Computing, Communication and Applied Informatics (ACCAI),IEEE, Chennai, India, 1-4. https://doi.org/10.1109/ACCAI61061.2024.10602097
  13. S.K. Trivedi, A study on credit scoring modelling with different feature selection and machine learning approaches, Technology in Society, 63, (2020) 1-9, https://doi.org/10.1016/j.techsoc.2020.101413
  14. M.Z. Abedin, C. Guotai, P. Hajek, T. Zhang, Combining weighted SMOTE with ensemble learning for the class-imbalanced prediction of small business credit risk. Complex & Intelligent Systems, 9(4), (2023) 3559-3579. https://doi.org/10.1007/s40747-021-00614-4
  15. M.S. Irfan Ahmed, P. Ramila Rajaleximi, A detailed analysis on classification algorithms for imbalanced class distribution on credit score datasets, Adalya Journal, 9(7), (2020) 244-251, 2020. https://doi.org/10.37896/aj9.7/023
  16. J. Xiao, Y. Wang, J. Chen, L. Xie, J. Huang, Impact of resampling methods and classification models on the imbalanced credit scoring problems, Information Sciences, 569, (2021) 508-526. https://doi.org/10.1016/j.ins.2021.05.029
  17. Z. Zhao, T. Cui, S. Ding, J. Li, A.G. Bellotti, Resampling techniques study on class imbalance problem in credit risk prediction, Mathematics, 12(5), (2024) 1-27. https://doi.org/10.3390/math12050701
  18. H. He, W. Zhang, S. Zhang, A novel ensemble method for credit scoring: Adaption of different imbalance ratios, Expert Systems with Applications, 98, (2018) 105-117. https://doi.org/10.1016/j.eswa.2018.01.012
  19. J. Abellán, J.G. Castellano, A comparative study on base classifiers in ensemble methods for credit scoring, Expert systems with applications, 73, (2017) 1-10. https://doi.org/10.1016/j.eswa.2016.12.020
  20. W. Zhang, H. He, S. Zhang, A novel multi-stage hybrid model with enhanced multi-population niche genetic algorithm: An application in credit scoring, Expert Systems with Applications, 121, (2019) 221-232. https://doi.org/10.1016/j.eswa.2018.12.020
  21. Luo, A comparison analysis for credit scoring using bagging ensembles, Expert Systems, 39(2), (2022) 1-7. https://doi.org/10.1111/exsy.12297
  22. L. Zhou, K.K. Lai, Adaboosting neural networks for credit scoring, In 2009 Proc. International Symposium on Neural Networks, Springer Berlin Heidelberg, 875-884. https://doi.org/10.1007/978-3-642-01216-7_93
  23. I. Ahmed, Credit risk management using hybrid scoring strategy and ensemble learning, Thesis Dissertation, Bharathiar University, India, 2020. http://hdl.handle.net/10603/397840
  24. Wang, Z. Zhang, R. Bai, Y. Mao, A hybrid system with filter approach and multiple population genetic algorithm for feature selection in credit scoring, Journal of Computational and Applied Mathematics, 329, (2018) 307-321. https://doi.org/10.1016/j.cam.2017.04.036
  25. Theng, K.K. Bhoyar, Feature selection techniques for machine learning: a survey of more than two decades of research. Knowledge and Information Systems, 66(3), (2024) 1575-1637. https://doi.org/10.1007/s10115-023-02010-5
  26. S. Sathya Bama, M.S. Irfan Ahmed, A. Saravanan, Relevance re-ranking through proximity based term frequency model, In ICT Innovations 2016: Cognitive Functions and Next Generation ICT Systems, Springer International Publishing, 219-229. https://doi.org/10.1007/978-3-319-68855-8_22
  27. S. Sathya Bama, M.I. Ahmed, A. Saravanan, A mathematical approach for mining web content outliers using term frequency ranking, Indian Journal of Science and Technology, 8(14), (2015) 1-5. https://dx.doi.org/10.17485/ijst/2015/v8i14/55679
  28. X. Cui, Y. Li, J. Fan, T. Wang, A novel filter feature selection algorithm based on relief, Applied Intelligence, 52(5), (2022) 5063-5081. https://doi.org/10.1007/s10489-021-02659-x
  29. W. Bouaguel, G. Bel Mufti, M. Limam, Rank aggregation for filter feature selection in credit scoring, In Proc. 2013 International Conference on Mining Intelligence and Knowledge Exploration, Springer International Publishing, 7-15. https://doi.org/10.1007/978-3-319-03844-5_2
  30. F.N. Koutanaei, H. Sajedi, M. Khanbabaei, A hybrid data mining model of feature selection algorithms and ensemble learning classifiers for credit scoring, Journal of Retailing and Consumer Services, 27, (2015) 11-23, https://doi.org/10.1016/j.jretconser.2015.07.003
  31. Boughaci, A.A.S. Alkhawaldeh, Three local search-based methods for feature selection in credit scoring, Vietnam Journal of Computer Science, 5, (2018), 107-121. https://doi.org/10.1007/s40595-018-0107-y
  32. P. Rajaleximi, M. Ahmed, A. Alenezi, Feature selection using optimized multiple rank score model for credit scoring, International Journal of Intelligent Engineering and Systems, 12(2), (2019) 74-84.
  33. D. Tripathi, D.R. Edla, R. Cheruku, V. Kuppili, A novel hybrid credit-scoring model based on ensemble feature selection and multilayer ensemble classification, Computational Intelligence, 35(2), (2019) 371-394. https://doi.org/10.1111/coin.12200
  34. S.F. Crone, S. Finlay, Instance sampling in credit scoring: An empirical study of sample size and balancing, International Journal of Forecasting, 28(1), (2012) 224-238. https://doi.org/10.1016/j.ijforecast.2011.07.006
  35. T. Jo, N. Japkowicz, Class imbalances versus small disjuncts, ACM Sigkdd Explorations Newsletter, 6(1), (2004) 40-49. https://doi.org/10.1145/1007730.1007737
  36. S.J. Bennehalli, S. Vakkund, A. Hegde, B. Bhowmik, Navigating data imbalances in credit risk management: A one-sided selection approach. In IEEE 2024 Control Instrumentation System Conference, IEEE, Manipal, India, 1-6. https://doi.org/10.1109/CISCON62171.2024.10696124
  37. N.V. Chawla, K.W. Bowyer, L.O. Hall, W.P. Kegelmeyer, SMOTE: synthetic minority over-sampling technique, Journal of artificial intelligence research, 16, (2002) 321-357. https://doi.org/10.1613/jair.953
  38. D.L. Wilson, Asymptotic properties of nearest neighbor rules using edited data, IEEE Transactions on Systems, Man, and Cybernetics, 3, (1972), 408-421. https://doi.org/10.1109/TSMC.1972.4309137
  39. G.E. Batista, R.C. Prati, M.C. Monard, A study of the behavior of several methods for balancing machine learning training data, ACM SIGKDD explorations newsletter, 6(1), (2004) 20-29. https://doi.org/10.1145/1007730.1007735
  40. H. He, B. Yang, E.A. Garcia, S.A. Li, Adaptive synthetic sampling approach for imbalanced learning, In Proc. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China.1322-1328.
  41. C. Bunkhumpornpat, K. Sinapiromsaran, C. Lursinsap, Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem, In Proc. 2009 Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, Bangkok, Thailand, Springer Berlin Heidelberg, 475-482. https://doi.org/10.1007/978-3-642-01307-2_43
  42. S. Barua, M.M. Islam, X. Yao, K. Murase, MWMOTE--majority weighted minority oversampling technique for imbalanced data set learning, IEEE Transactions on knowledge and data engineering, IEEE, 26(2), (2012) 405-425. https://doi.org/10.1109/TKDE.2012.232
  43. Bao, Y. Wu, Z. Li, Y. Li, L. Liu, G. Chen, Effect improved for high-dimensional and unbalanced data anomaly detection model based on KNN-SMOTE-LSTM, Complexity, 2020(1), (2020) 9084704. https://doi.org/10.1155/2020/9084704
  44. Han, W.Y. Wang, B.H. Mao, Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning, In Proc. 2005 International Conference on Advances in Intelligent Computing, China, Springer Berlin Heidelberg, 878-887. https://doi.org/10.1007/11538059_91
  45. V. García, A.I. Marqués, J.S. Sánchez, Improving risk predictions by preprocessing imbalanced credit data, In Proc. 2012 International Conference on Neural Information Processing, Doha, Qatar, Springer Berlin Heidelberg, 68-75. https://doi.org/10.1007/978-3-642-34481-7_9
  46. C. Jiang, W. Lu, Z. Wang, Y. Ding, Benchmarking state-of-the-art imbalanced data learning approaches for credit scoring, Expert systems with applications, 213(B), (2023) 118878. https://doi.org/10.1016/j.eswa.2022.118878
  47. Y. Xia, C. Liu, B. Da, F. Xie, A novel heterogeneous ensemble credit-scoring model based on bstacking approach, Expert Systems with Applications, 93, (2018) 182-199. https://doi.org/10.1016/j.eswa.2017.10.022
  48. M. Abdoli, M. Akbari, J. Shahrabi, Bagging supervised autoencoder classifier for credit scoring, Expert Systems with Applications, 213, (2023) 118991. https://doi.org/10.1016/j.eswa.2022.118991
  49. R.E. Schapire, The strength of weak learnability, Machine learning, 5, (1990) 197-227. https://doi.org/10.1007/BF00116037
  50. D.H. Wolpert, Stacked generalization, Neural networks, 5(2), (1992) 241-259. https://doi.org/10.1016/S0893-6080(05)80023-1
  51. F.M. Talaat, A. Aljadani, M. Badawy, M. Elhosseini, Toward interpretable credit scoring: integrating explainable artificial intelligence with deep learning for credit card default prediction, Neural Computing and Applications, 36(9), (2024) 4847-4865. https://doi.org/10.1007/s00521-023-09232-2
  52. A.V. Chaudhari, P.A. Charate, Synthetic Financial Document Generation and Fraud Detection Using Generative AI and Explainable ML, Journal of Recent Trends in Computer Science and Engineering, 13(2), (2025) 45-59. https://doi.org/10.70589/JRTCSE.2025.13.2.6
  53. M. Ala'raj, M.F. Abbod, Classifiers consensus system approach for credit scoring, Knowledge-Based Systems, 104, (2016) 89-105. https://doi.org/10.1016/j.knosys.2016.04.013
  54. S. Jadhav, H. He, K. Jenkins, Information gain directed genetic algorithm wrapper feature selection for credit rating, Applied Soft Computing, 69, (2018) 541-553. https://doi.org/10.1016/j.asoc.2018.04.033
  55. Y. Hayashi, T. Oishi, High accuracy-priority rule extraction for reconciling accuracy and interpretability in credit scoring, New Generation Computing, 36(4), (2018) 393-418. https://doi.org/10.1007/s00354-018-0043-5
  56. F. Shen, X. Zhao, Z. Li, K. Li, Z. Meng, A novel ensemble classification model based on neural networks and a classifier optimisation technique for imbalanced credit risk evaluation, Physica A: Statistical Mechanics and its Applications, 526, (2019) 121073. https://doi.org/10.1016/j.physa.2019.121073
  57. J. Xiao, Y. Wang, J. Chen, L. Xie, J. Huang, Impact of resampling methods and classification models on the imbalanced credit scoring problems. Information Sciences, 569, (2021) 508-526. https://doi.org/10.1016/j.ins.2021.05.029
  58. S.S. Bama, A. Saravanan, Efficient classification using average weighted pattern score with attribute rank based feature selection, International Journal of Intelligent Systems and Applications, 11(7), (2019) 29-42. https://doi.org/10.5815/ijisa.2019.07.04
  59. M. Owusu-Adjei, J. Ben Hayfron-Acquah, T. Frimpong, G. Abdul-Salaam, Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems, PLOS Digital Health, 2(11), (2023) e0000290. https://doi.org/10.1371/journal.pdig.0000290
  60. D. Chicco, N. Tötsch, G. Jurman, The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation, BioData mining, 14(1), (2021) 13. https://doi.org/10.1186/s13040-021-00244-z
  61. M. Li, Q. Gao, T. Yu, Kappa statistic considerations in evaluating inter-rater reliability between two raters: which, when and context matters, BMC cancer, 23(1), (2023) 799. https://doi.org/10.1186/s12885-023-11325-z
  62. V. García, R.A. Mollineda, J.S. Sánchez, Index of balanced accuracy: A performance measure for skewed class distributions, In Proc. 2009 Iberian Conference on Pattern Recognition and Image Analysis, Portugal, Springer Berlin Heidelberg, (2009) 441-448. https://doi.org/10.1007/978-3-642-02172-5_57
  63. S. Lessmann, B. Baesens, H.V. Seow, L.C. Thomas, Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research, European Journal of Operational Research, 247(1), (2015) 124-136. https://doi.org/10.1016/j.ejor.2015.05.030