Abstract

Toxic online content poses significant challenges to digital communication platforms, necessitating accurate and balanced classification strategies. Unlike traditional binary classification approaches, this study focuses on multi-label toxic comment classification using the Jigsaw dataset, where each comment may exhibit multiple overlapping toxicity types. To address the severe class imbalance inherent in the dataset, three tailored undersampling strategies—One-vs-Rest Undersampling, Multilabel Random Undersampling (MLRU), and an Improved Threshold-Based Undersampling—are proposed to enhance the representation of minority labels such as threat and identity hate. These undersampled datasets are evaluated using a diverse set of ML and DL models, including Random Forest, XGBoost, CNN, RNN, LSTM, BiLSTM, BERT, and RoBERTa. Experimental results shown that this joint multi-label–specific under sampling combined with advanced classification architectures establish superior models, more evident in early detection of rare but relevant types of toxicity. This work demonstrates that multi-label learning frameworks can serve as an effective approach toward fair and full toxic comment detection.

Keywords

Multi-Label Classification, Under sampling, Deep Learning, Transformer Models, Toxic Comment Classification,

Downloads

Download data is not yet available.

References

  1. J. A. Diaz‐Garcia, J.P. Carvalho, A Literature Review of Textual Cyber Abuse Detection Using Cutting‐Edge Natural Language Processing Techniques: Language Models and Large Language Models. WIREs Data Mining and Knowledge Discovery, 15(3), (2025) e70029. https://doi.org/10.1002/widm.70029
  2. M.S. Jahan, M. Oussalah, A systematic review of hate speech automatic detection using natural language processing. Neurocomputing, 546, (2023) 126232. https://doi.org/10.1016/j.neucom.2023.126232
  3. A. Rashid, S. Mahmood, U. Inayat, M.F. Zia, Urdu Toxicity Detection: A Multi-Stage and Multi-Label Classification Approach, AI, 6(8), (2025) 194. https://doi.org/10.3390/ai6080194
  4. M. Neog, N. Baruah, A hybrid deep learning approach for Assamese toxic comment detection in social media. Procedia Computer Science, 235, (2024) 2297–2306. https://doi.org/10.1016/j.procs.2024.04.218
  5. F. Charte, A.J. Rivera, M.J. del Jesus, F. Herrera, Addressing imbalance in multilabel classification: Measures and random resampling algorithms, Neurocomputing, 163, (2015) 3–16. https://doi.org/10.1016/j.neucom.2014.08.091
  6. M. Xu, S. Liu, RB_BG_MHA: A RoBERTa-Based Model with Bi-GRU and Multi-Head Attention for Chinese Offensive Language Detection in Social Media. Applied Sciences, 13(19), (2023) 11000. https://doi.org/10.3390/app131911000
  7. S.K. Putri, A. Amalia, T.F. Abidin. (2024) Sentiment analysis multi-label of toxic comments using BERT-BiLSTM methods. International Conference on Electrical Engineering and Informatics (ICELTICs), 2024, IEEE, Banda Aceh, Indonesia, 120-124. https://doi.org/10.1109/ICELTICs62730.2024.10776338
  8. A. Abbasi, A.R. Javed, F. Iqbal, N. Kryvinska, Z. Jalil, Deep learning for religious and continent-based toxic content detection and classification. Scientific Reports, 12(1), (2022) 17478. https://doi.org/10.1038/s41598-022-22523-3
  9. K. Tejwani, V. Naik, A. Lari, D. Jhaveri. (2024) Enhancing toxic comment classification: A deep learning approach with pre-trained language models. 2024 International Conference on Intelligent Systems and Advanced Applications (ICISAA), Pune, India, 1–6, https://doi.org/10.1109/ICISAA62385.2024.10829297
  10. S. Awasthi, S. K. Shukla, D. Sharma, D. Gupta, A. Tripathi. (2025) Combating cyber abuse: A toxic comment detection model using deep learning. In 2025 3rdInternational Conference on Communication, Security, and Artificial Intelligence (ICCSAI), Greater Noida, India. https://doi.org/10.1109/ICCSAI64074.2025.11064521
  11. Y. Sagama, A. Alamsyah. (2023) Multi-Label Classification of Indonesian Online Toxicity using BERT and RoBERTa. 2023 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT). IEEE, BALI, Indonesia, 143–149. https://doi.org/10.1109/IAICT59002.2023.10205892
  12. S. Sushma, S.K. Nayak, M.V. Krishna, (2024) A Comprehensive Review of Sentiment Analysis: Trends, Challenges, and Future Directions. 2024 5th International Conference on Data Intelligence and Cognitive Informatics (ICDICI), Tirunelveli, India, 1175-1181. https://doi.org/10.1109/ICDICI62993.2024.10810919
  13. L.G. Atlas, D. Arockiam, A. Muthusamy, B. Balusamy, S. Selvarajan, T. Al-Shehari, N.A. Alsadhan, A modernized approach to sentiment analysis of product reviews using BiGRU and RNN based LSTM deep learning models. Scientific Reports, 15(1), (2025) 16642. https://doi.org/10.1038/s41598-025-01104-0
  14. Y. Cai, X. Li, Y. Zhang, J. Li, F. Zhu, L. Rao, Multimodal sentiment analysis based on multi-layer feature fusion and multi-task learning. Scientific Reports, 15(1), (2025) 2126. https://doi.org/10.1038/s41598-025-85859-6
  15. S.M. Ferdous, S.N.E. Newaz, S.B.S. Mugdha, M. Uddin, Sentiment Analysis in the Transformative Era of Machine Learning: A Comprehensive Review. Statistics, Optimization & Information Computing, 13(1), (2024) 331–346. https://doi.org/10.19139/soic-2310-5070-2113
  16. N.A. Semary, W. Ahmed, K. Amin, P. Pławiak, M. Hammad, Enhancing machine learning-based sentiment analysis through feature extraction techniques. PLoS ONE, 19(2), (2024) e0294968. https://doi.org/10.1371/journal.pone.0294968
  17. N. Punetha, G. Jain, Advancing sentiment classification through a population game model approach. Scientific Reports, 14(1), (2024) 20540. https://doi.org/10.1038/s41598-024-70766-z
  18. P.D. Michailidis, A Comparative Study of Sentiment Classification Models for Greek Reviews. Big Data and Cognitive Computing, 8(9), (2024) 107. https://doi.org/10.3390/bdcc8090107
  19. S. Rezaei, J. Tanha, S. Roshan, Z. Jafari, M. Molaei, S. Mirzadoust, M. Sadeghi, A. Forsati, T. Khoshamouz. An experimental study of sentiment classification using deep-based models with various word embedding techniques. Journal of Experimental & Theoretical Artificial Intelligence, (2024) 1–37. https://doi.org/10.1080/0952813X.2024.2384568
  20. J. Jose, R. Simritha, Sentiment Analysis and Topic Classification with LSTM Networks and TextRazor. International Journal of Data Informatics and Intelligent Computing, 3(2), (2024) 42–51. https://doi.org/10.59461/ijdiic.v3i2.115
  21. M. Ijaz, N. Anwar, M. Safran, S. Alfarhood, T. Sadad, Imran. Domain adaptive learning for multi realm sentiment classification on big data. PLoS ONE, 19(4), (2024) e0297028. https://doi.org/10.1371/journal.pone.0297028
  22. A. Lakshmanarao, C. Gupta, T.S.R. Kiran. (2022) Airline Twitter Sentiment Classification using Deep Learning Fusion. In 2022 International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON), Bangalore, India, 1-4. https://doi.org/10.1109/SMARTGENCON56628.2022.10084207
  23. A.S. Talaat, Sentiment analysis classification system using hybrid BERT models. Journal of Big Data, 10(1), (2023) 110. https://doi.org/10.1186/s40537-023-00781-w
  24. S. Sushma, S.K. Nayak, M.V. Krishna. (2025) An Efficient Toxic Comment Classification using Hybrid Machine Learning Algorithms with TF-IDF and Word2Vec Word Embeddings. In 2025 Third International Conference on Augmented Intelligence and Sustainable Systems (ICAISS), Trichy, India, 1416-1421. https://doi.org/10.1109/ICAISS61471.2025.11042068
  25. A. Alsharef, K. Aggarwal, Sonia, D. Koundal, H. Alyami, D. Ameyed. An Automated Toxicity Classification on Social Media Using LSTM and Word Embedding. Computational Intelligence and Neuroscience, 2022(1), (2022) 8467349. https://doi.org/10.1155/2022/8467349
  26. A. Khan, M.A. Qureshi, B. Mondal. Sentiment analysis of emoji fused reviews using machine learning and Bert. Scientific Reports, 15(1), (2025) 7538. https://doi.org/10.1038/s41598-025-92286-0
  27. E. Elbasani, J.D. Kim, AMR-CNN: Abstract Meaning Representation with Convolution Neural Network for Toxic Content Detection. In Journal of Web Engineering, 21(3), (2022) 677-692. https://doi.org/10.13052/jwe1540-9589.2135
  28. Abhishek Aggarwal, Atul Tiwari. Multi Label Toxic Comment Classification using Machine Learning Algorithms. International Journal of Recent Technology and Engineering (IJRTE), 10 (1), (2021) 158-161. http://www.doi.org/10.35940/ijrte.A5814.0510121
  29. W. Guo, J. Liu, F. Dong, M. Song, Z. Li, M.K.H. Khan, T.A. Patterson, H. Hong. Review of machine learning and deep learning models for toxicity prediction. Experimental Biology and Medicine, 248(21), (2023) 1952-1973.
  30. J. Jotheeswaran, V. Geetha, M. Iyyappan, K.G. Srinivasa. (2025) Promoting Constructive Online Debates through Toxic Comment Classifier, 2024 International Conference on IT Innovation and Knowledge Discovery (ITIKD), Manama, Bahrain, 1-6. https://doi.org/10.1109/ITIKD63574.2025.11004955
  31. R. Musonzo, (2025) Toxic Comment Classification Using Deep Learning. Elsevier BV. https://dx.doi.org/10.2139/ssrn.5192997
  32. S. Sushma, Sasmita Kumari Nayak, and M. V. Krishna. Enhanced toxic comment detection model through Deep Learning models using Word embeddings and transformer architectures. futech, 4(3), (2025) 76–84. https://doi.org/10.55670/fpll.futech.4.3.8
  33. S. Robinson. (2023) Classification of Toxic Comments Based on Textual Data Using Deep Learning Algorithms. Available at SSRN 4609428. https://dx.doi.org/10.2139/ssrn.4609428
  34. B.S. Lakshmi, T. Shravya, L. Yaswitha, S. Shahin, V. Vijayadeepa. (2025) Toxinet: A Deep Learning Framework for Online Comment Toxicity Detection, 2025 International Conference on Inventive Computation Technologies (ICICT). IEEE, Kirtipur, Nepal, 1–6. https://doi.org/10.1109/ICICT64420.2025.11005096
  35. A. Bonetti, M. Martínez-Sober, J.C. Torres, J.M. Vega, S. Pellerin, J. Vila-Francés. Comparison between Machine Learning and Deep Learning Approaches for the Detection of Toxic Comments on Social Networks. Applied Sciences, 13(10), (2023) 6038. https://doi.org/10.3390/app13106038
  36. A.K. Pal, S. Rai. (2023) Toxicity Tweet Detection and Classification Using NLP Driven Techniques. In 2023 IEEE International Conference on ICT in Business Industry & Government (ICTBIG), IEEE, Indore, India, 1–4, https://doi.org/10.1109/ICTBIG59752.2023.10456026
  37. S. Sushma, S.K. Nayak, M.V. Krishna. Advanced Toxic Comment Classification Using Multi-‎Architecture Generative AI Techniques. International Journal of Basic and Applied Sciences, 14(4), (2025) 499–507.
  38. https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/data
  39. S. Pradha, M.N. Halgamuge, N. Tran Quoc Vinh. (2019) Effective Text Data Preprocessing Technique for Sentiment Analysis in Social Media Data. In 2019 11th International Conference on Knowledge and Systems Engineering (KSE), Da Nang, Vietnam, 1-8. https://doi.org/10.1109/KSE.2019.8919368
  40. J. Devlin, M.W. Chang, K. Lee, K. Toutanova. (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. https://doi.org/10.48550/arXiv.1810.04805
  41. H.T. Duong, T.A. Nguyen Thi, A review: preprocessing techniques and data augmentation for sentiment analysis. Computational Social Networks, 8(1), (2021). https://doi.org/10.1186/s40649-020-00080-x
  42. V. Maslej-Krešňáková, M. Sarnovský, P. Butka, K. Machová, Comparison of deep learning models and various text pre-processing techniques for the toxic comments classification. Applied Sciences, 10(23), (2020) 8631. https://doi.org/10.3390/app10238631
  43. E. Bak, Y. An, S. Pan, (2023) A Novel Multi-Label Evaluation Measure with Comparative Analysis. International Conference on Machine Learning and Applications (ICMLA), IEEE, USA. https://doi.org/10.1109/ICMLA58977.2023.00080
  44. G. Nasierding, A.Z. Kouzani, (2012) Comparative evaluation of multi-label classification methods. 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, IEEE, China. https://doi.org/10.1109/FSKD.2012.6234347
  45. A.Y. Taha, S. Tiun, A.H. Abd Rahman, A. Sabah, Multilabel Over-Sampling and Under-Sampling with Class Alignment for Imbalanced Multilabel Text Classification. Journal of Information and Communication Technology, 20(3), (2021) 423–456. https://doi.org/10.32890/jict2021.20.3.6
  46. X. Gao, Y. He, M. Zhang, X. Diao, X. Jing, B. Ren, W. Ji, A multiclass classification using one-versus-all approach with the differential partition sampling ensemble. Engineering Applications of Artificial Intelligence, 97, (2021) 104034. https://doi.org/10.1016/j.engappai.2020.104034
  47. N. Boudjani, Y. Haralambous, I. Lyubareva, (2020) Toxic Comment Classification for French Online Comments. IEEE International Conference on Machine Learning and Applications (ICMLA), EEE, USA. https://doi.org/10.1109/ICMLA51294.2020.00164
  48. K.B. Nelatoori, H.B. Kommanti, Multi-task learning for toxic comment classification and rationale extraction. Journal of Intelligent Information Systems, 60(2), (2023) 495-519. https://doi.org/10.1007/s10844-022-00726-4
  49. H.H.P. Vo, H. Trung Tranm, S.T. Luu, (2021) Automatically Detecting Cyberbullying Comments on Online Game Forums. 2021 RIVF International Conference on Computing and Communication Technologies (RIVF), IEEE, Hanoi, Vietnam. https://doi.org/10.1109/RIVF51545.2021.9642116
  50. M.N. Fauzan, A.G. Putrada, N. Alamsyah, S.F. Pane, (2022) PCA-AdaBoost Method for a Low Bias and Low Dimension Toxic Comment Classification. International Conference on Advanced Creative Networks and Intelligent Systems (ICACNIS), IEEE, Bandung, Indonesia. https://doi.org/10.1109/ICACNIS57039.2022.10055017