International Journal of

ADVANCED AND APPLIED SCIENCES

EISSN: 2313-3724, Print ISSN: 2313-626X

Frequency: 12

line decor
  
line decor

 Volume 12, Issue 7 (July 2025), Pages: 55-75

----------------------------------------------

 Original Research Paper

A holistic evaluation of machine learning algorithms for text-based emotion detection

 Author(s): 

 Syed Zafar Ali Shah 1, Omar Ahmed Abdulkader 2, Sadaqat Jan 3, Muhammad Arif Shah 4, Muhammad Anwar 5, *

 Affiliation(s):

  1Department of Computer Software Engineering, University of Engineering and Technology, Peshawar, Peshawar, 25120, Pakistan
  2Department of Computer Studies, Arab Open University, Riyadh, Saudi Arabia
  3Department of Computer Software Engineering, University of Engineering and Technology, Mardan, 23200, Pakistan
  4Department of IT and Computer Science, Pak-Austria Fachhochschule Institute of Applied Sciences and Technology, Haripur, Pakistan
  5Department of Information Sciences, Division of Science and Technology, University of Education, Lahore, 54000, Pakistan

 Full text

    Full Text - PDF

 * Corresponding Author. 

   Corresponding author's ORCID profile:  https://orcid.org/0000-0002-0615-3038

 Digital Object Identifier (DOI)

  https://doi.org/10.21833/ijaas.2025.07.006

 Abstract

The rapid growth of social media and text-based communication has intensified interest in emotion detection (ED) from text. Extracting emotional content from large-scale textual sources—such as social media posts, blogs, and news articles—is both challenging and critical for various applications. This study evaluates the effectiveness of traditional machine learning algorithms in text-based emotion detection by conducting a systematic literature review (SLR), expert-based evaluation, and multiple case studies. The SLR, based on seven major digital libraries, applied a five-phase selection process to identify the most relevant studies. Findings show that Support Vector Machine (SVM) is the most frequently used and top-performing model (78% of studies), followed by Naive Bayes (60%), with customized datasets preferred in 70% of the literature. The Ekman model with six emotion classes was the most common framework, while datasets with four to eight emotion categories yielded the highest accuracy. An Analytical Hierarchy Process (AHP) involving 82 industry experts ranked SVM highest in accuracy, robustness, and interpretability, followed by Naive Bayes and Random Forest. Case studies further confirmed the strong performance of SVM, Logistic Regression, and Naive Bayes, with ensemble models improving accuracy by 3% over the best individual classifier. Additionally, the study explores transformer-based models, finding that DeBERTa outperforms traditional approaches by better capturing emotional subtleties in text. Limitations of conventional models are discussed, and practical recommendations for future improvements are provided.

 © 2025 The Authors. Published by IASE.

 This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/).

 Keywords

 Emotion detection, Machine learning, Text analysis, Transformer models, Sentiment classification

 Article history

 Received 20 September 2024, Received in revised form 21 May 2025, Accepted 8 June 2025

 Acknowledgment

The authors extend their appreciation to the Arab Open University for funding this work through the AOU research fund No. (AOUKSA524008). 

 Compliance with ethical standards

 Ethical considerations

The authors confirm that participation in the expert survey was voluntary. Informed consent was obtained from all participants prior to data collection. No personal or sensitive information was collected, and all responses were anonymized to ensure confidentiality.

 Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

 Citation:

 Shah SZA, Abdulkader OA, Jan S, Shah MA, and Anwar M (2025). A holistic evaluation of machine learning algorithms for text-based emotion detection. International Journal of Advanced and Applied Sciences, 12(7): 55-75

  Permanent Link to this page

 Figures

  Fig. 1  Fig. 2  Fig. 3  Fig. 4  Fig. 5  Fig. 6  Fig. 7  Fig. 8  Fig. 9  Fig. 10  Fig. 11  Fig. 12  Fig. 13  Fig. 14  Fig. 15  Fig. 16  Fig. 17  Fig. 18 

 Tables

  Table 1  Table 2  Table 3  Table 4  Table 5  Table 6  Table 7  Table 8  Table 9  Table 10  Table 11  Table 12  Table 13  Table 14 

----------------------------------------------   

 References (63)

  1. Abdolvand N, Albadvi A, and Aghdasi M (2015). Performance management using a value-based customer-centered model. International Journal of Production Research, 53(18): 5472–5483.  https://doi.org/10.1080/00207543.2015.1026613    [Google Scholar]
  2. Abdullah M, AlMasawa M, Makki I, Alsolmi M, and Mahrous S (2020). Emotions extraction from Arabic tweets. International Journal of Computers and Applications, 42(7): 661–675.  https://doi.org/10.1080/1206212X.2018.1482395    [Google Scholar]
  3. Abrar MF, Khan MS, Khan I, Ali G, and Shah S (2023b). Digital information credibility: Towards a set of guidelines for quality assessment of grey literature in multivocal literature review. Applied Sciences, 13(7): 4483.  https://doi.org/10.3390/app13074483    [Google Scholar]
  4. Abrar MF, Khan MS, Khan I, ElAffendi M, and Ahmad S (2023a). Towards fake news detection: A multivocal literature review of credibility factors in online news stories and analysis using analytical hierarchical process. Electronics, 12(15): 3280.  https://doi.org/10.3390/electronics12153280    [Google Scholar]
  5. Acheampong FA, Wenyu C, and Nunoo-Mensah H (2020). Text-based emotion detection: Advances, challenges, and opportunities. Engineering Reports, 2(7): e12189.    [Google Scholar]
  6. Almahdawi A and Teahan WJ (2018). Automatically recognizing emotions in text using prediction by partial matching (PPM) text compression method. In the New Trends in Information and Communications Technology Applications: 3rd International Conference, Springer International Publishing, Baghdad, Iraq: 269-283.  https://doi.org/10.1007/978-3-030-01653-1_17    [Google Scholar]
  7. Alotaibi FM (2019). Classifying text-based emotions using logistic regression. VAWKUM Transactions on Computer Sciences, 7(1): 31–37.  https://doi.org/10.21015/vtcs.v16i2.551    [Google Scholar]
  8. Alswaidan N and Menai MEB (2020). A survey of state-of-the-art approaches for emotion recognition in text. Knowledge and Information Systems, 62(8): 2937-2987.  https://doi.org/10.1007/s10115-020-01449-0    [Google Scholar]
  9. Angel Deborah S, Rajalakshmi S, Milton Rajendram S, and Mirnalinee TT (2020). Contextual emotion detection in text using ensemble learning. In: Hemanth DJ, Kumar VDA, Malathi S, Castillo O, and Patrut B (Eds.), Emerging trends in computing and expert technology: 1179–1186. Springer, Cham, Switzerland.  https://doi.org/10.1007/978-3-030-32150-5_121    [Google Scholar]
  10. Assiri A, Gumaei A, Mehmood F, Abbas T, and Ullah S (2024). DeBERTa-GRU: Sentiment analysis for large language model. Computers, Materials and Continua, 79(3): 4219-4236.  https://doi.org/10.32604/cmc.2024.050781    [Google Scholar]
  11. Balakrishnan V and Kaur W (2019). String-based multinomial Naïve Bayes for emotion detection among Facebook diabetes community. Procedia Computer Science, 159: 30–37.  https://doi.org/10.1016/j.procs.2019.09.157    [Google Scholar]
  12. Balakrishnan V, Lok PY, and Abdul Rahim H (2021). A semi-supervised approach in detecting sentiment and emotion based on digital payment reviews. The Journal of Supercomputing, 77(4): 3795–3810.  https://doi.org/10.1007/s11227-020-03412-w    [Google Scholar]
  13. Carrera-Rivera A, Ochoa W, Larrinaga F, and Lasa G (2022). How-to conduct a systematic literature review: A quick guide for computer science research. MethodsX, 9: 101895.  https://doi.org/10.1016/j.mex.2022.101895    [Google Scholar] PMid:36405369 PMCid:PMC9672331
  14. Chaffar S and Inkpen D (2011). Using a heterogeneous dataset for emotion analysis in text. In the Advances in Artificial Intelligence: 24th Canadian Conference on Artificial Intelligence, Canadian AI 2011, Springer Berlin Heidelberg, St. John’s, Canada: 62-67.  https://doi.org/10.1007/978-3-642-21043-3_8    [Google Scholar]
  15. Chakriswaran P, Vincent DR, Srinivasan K, Sharma V, Chang CY, and Reina DG (2019). Emotion AI-driven sentiment analysis: A survey, future research directions, and open issues. Applied Sciences, 9(24): 5462.  https://doi.org/10.3390/app9245462    [Google Scholar]
  16. Chan YL, Bea KT, Leow SMH, Phoong SW, and Cheng WK (2023). State of the art: A review of sentiment analysis based on sequential transfer learning. Artificial Intelligence Review, 56(1): 749–780.  https://doi.org/10.1007/s10462-022-10183-8    [Google Scholar]
  17. Chowanda A, Sutoyo R, and Tanachutiwat S (2021). Exploring text-based emotions recognition machine learning techniques on social media conversation. Procedia Computer Science, 179: 821-828.  https://doi.org/10.1016/j.procs.2021.01.099    [Google Scholar]
  18. Esmin AA, De Oliveira Jr RL, and Matwin S (2012). Hierarchical classification approach to emotion recognition in Twitter. In the 11th International Conference on Machine Learning and Applications, IEEE, Boca Raton, USA, 2: 381-385.  https://doi.org/10.1109/ICMLA.2012.195    [Google Scholar]
  19. Ghanbari-Adivi F and Mosleh M (2019). Text emotion detection in social networks using a novel ensemble classifier based on Parzen tree estimator (TPE). Neural Computing and Applications, 31(12): 8971-8983.  https://doi.org/10.1007/s00521-019-04230-9    [Google Scholar]
  20. Gohil L and Patel D (2019). Multilabel classification for emotion analysis of multilingual tweets. International Journal of Innovative Technology and Exploring Engineering, 9(1): 4453–4457.  https://doi.org/10.35940/ijitee.A5320.119119    [Google Scholar]
  21. Gunarathne SR, De Silva J, Ekanayake EP, Samaradiwakara I, Haddela PS, and Fernando PA (2013). Intellemo: A mobile instant messaging application with intelligent emotion identification. In the IEEE 8th International Conference on Industrial and Information Systems, IEEE, Peradeniya, Sri Lanka: 627-632.  https://doi.org/10.1109/ICIInfS.2013.6732057    [Google Scholar]
  22. Halczak P (2023). Dictionary representation of the semantics of adjectives signifying emotions. International Journal of Lexicography, 36(4): 424–446.  https://doi.org/10.1093/ijl/ecad016    [Google Scholar]
  23. Halim Z, Waqar M, and Tahir M (2020). A machine learning-based investigation utilizing the in-text features for the identification of dominant emotion in an email. Knowledge-Based Systems, 208: 106443.  https://doi.org/10.1016/j.knosys.2020.106443    [Google Scholar]
  24. Hussein A, Al Kafri M, Abonamah AA, and Tariq MU (2020). Mood detection based on Arabic text documents using machine learning methods. International Journal, 9(4): 4424-4436.  https://doi.org/10.30534/ijatcse/2020/36942020    [Google Scholar]
  25. Jain VK, Kumar S, and Fernandes SL (2017). Extraction of emotions from multilingual text using intelligent text processing and computational linguistics. Journal of Computational Science, 21: 316-326.  https://doi.org/10.1016/j.jocs.2017.01.010    [Google Scholar]
  26. Kabra G, Ramesh A, and Arshinder K (2015). Identification and prioritization of coordination barriers in humanitarian supply chain management. International Journal of Disaster Risk Reduction, 13: 128-138.  https://doi.org/10.1016/j.ijdrr.2015.01.011    [Google Scholar]
  27. Kalcheva N, Karova M, and Penev I (2020). Comparison of the accuracy of SVM kemel functions in text classification. In the International Conference on Biomedical Innovations and Applications (BIA), IEEE, Varna, Bulgaria: 141-145.  https://doi.org/10.1109/BIA50171.2020.9244278    [Google Scholar]
  28. Kang X, Ren F, and Wu Y (2017). Exploring latent semantic information for textual emotion recognition in blog articles. IEEE/CAA Journal of Automatica Sinica, 5(1): 204–216.  https://doi.org/10.1109/JAS.2017.7510421    [Google Scholar]
  29. Kaur W, Balakrishnan V, and Singh B (2020). Improving teaching and learning experience in engineering education using sentiment analysis techniques. IOP Conference Series: Materials Science and Engineering, 834(1): 12026.  https://doi.org/10.1088/1757-899X/834/1/012026    [Google Scholar]
  30. Krenicky T, Hrebenyk L, and Chernobrovchenko V (2022). Application of concepts of the analytic hierarchy process in decision-making. Management Systems in Production Engineering, 4(30): 304-310.  https://doi.org/10.2478/mspe-2022-0039    [Google Scholar]
  31. Kumar A, Nadeem M, and Shameem M (2023). Systematic literature review of metrics for measuring devops success. AIP Conference Proceedings, 2724(1): 030005.  https://doi.org/10.1063/5.0128883    [Google Scholar]
  32. Liu L and Qi J (2018). Research on discrete emotion classification of Chinese online product reviews based on OCC model. In the IEEE 3rd International Conference on Data Science in Cyberspace, IEEE, Guangzhou, China: 371-378.  https://doi.org/10.1109/DSC.2018.00060    [Google Scholar]
  33. Mahajan R and Zaveri M (2021). Harnessing emotive features for emotion recognition from text. International Journal of Advanced Computer Science and Applications, 12(7): 166-175.  https://doi.org/10.14569/IJACSA.2021.0120719    [Google Scholar]
  34. Majeed A, Mujtaba H, and Beg MO (2020). Emotion detection in Roman Urdu text using machine learning. In the Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, ACM, Virtual Event, Australia: 125-130.  https://doi.org/10.1145/3417113.3423375    [Google Scholar]
  35. Mondal A and Gokhale SS (2020). Mining emotions on Plutchik's wheel. In the 7th International Conference on Social Networks Analysis, Management and Security, IEEE, Paris, France: 1-6.  https://doi.org/10.1109/SNAMS52053.2020.9336534    [Google Scholar]
  36. Munezero M, Montero CS, Sutinen E, and Pajunen J (2014). Are they different? Affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Transactions on Affective Computing, 5(2): 101–111.  https://doi.org/10.1109/TAFFC.2014.2317187    [Google Scholar]
  37. Murthy AR and Kumar KMA (2021). A review of different approaches for detecting emotion from text. IOP Conference Series: Materials Science and Engineering, 1110(1): 12009.  https://doi.org/10.1088/1757-899X/1110/1/012009    [Google Scholar]
  38. Nandwani P and Verma R (2021). A review on sentiment analysis and emotion detection from text. Social Network Analysis and Mining, 11(1): 81.  https://doi.org/10.1007/s13278-021-00776-6    [Google Scholar] PMid:34484462 PMCid:PMC8402961
  39. Nguyen KP-Q and Van Nguyen K (2020). Exploiting Vietnamese social media characteristics for textual emotion recognition in Vietnamese. In the International Conference on Asian Language Processing (IALP): 276–281.  https://doi.org/10.1109/IALP51396.2020.9310495    [Google Scholar]
  40. Pang J, Rao Y, Xie H, Wang X, Wang FL, Wong T-L, and Li Q (2019). Fast supervised topic models for short text emotion detection. IEEE Transactions on Cybernetics, 51(2): 815–828.  https://doi.org/10.1109/TCYB.2019.2940520    [Google Scholar] PMid:31567111
  41. Parvin T and Hoque MM (2021). An ensemble technique to classify multi-class textual emotion. Procedia Computer Science, 193: 72–81.  https://doi.org/10.1016/j.procs.2021.10.008    [Google Scholar]
  42. Patacsil FF (2020). Emotion recognition from blog comments based automatically generated datasets and ensemble models. International Journal, 9(4): 5979-5986.  https://doi.org/10.30534/ijatcse/2020/264942020    [Google Scholar]
  43. Patil T and Patil S (2013). Automatic generation of emotions for social networking websites using text mining. In the 4th International Conference on Computing, Communications and Networking Technologies, IEEE, Tiruchengode, India: 1-6.  https://doi.org/10.1109/ICCCNT.2013.6726704    [Google Scholar]
  44. Paul J and Barari M (2022). Meta‐analysis and traditional systematic literature reviews—What, why, when, where, and how? Psychology and Marketing, 39(6): 1099–1115.  https://doi.org/10.1002/mar.21657    [Google Scholar]
  45. Plaza-del-Arco FM, Martín-Valdivia MT, Urena-Lopez LA, and Mitkov R (2020). Improved emotion recognition in Spanish social media through incorporation of lexical knowledge. Future Generation Computer Systems, 110: 1000-1008.  https://doi.org/10.1016/j.future.2019.09.034    [Google Scholar]
  46. Povoda L, Arora A, Singh S, Burget R, and Dutta MK (2015). Emotion recognition from helpdesk messages. In the 7th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, IEEE, Brno, Czech Republic: 310-313.  https://doi.org/10.1109/ICUMT.2015.7382448    [Google Scholar]
  47. Povoda L, Burget R, Masek J, Uher V, and Dutta MK (2016). Optimization methods in emotion recognition system. Radioengineering, 25(3): 565–572.  https://doi.org/10.13164/re.2016.0565    [Google Scholar]
  48. Putra OV, Wasmanson FM, Harmini T, and Utama SN (2020). Sundanese Twitter dataset for emotion classification. In the International Conference on Computer Engineering, Network, and Intelligent Multimedia, IEEE, Surabaya, Indonesia: 391-395.  https://doi.org/10.1109/CENIM51130.2020.9297929    [Google Scholar]
  49. Ray A, Bala PK, and Jain R (2021). Utilizing emotion scores for improving classifier performance for predicting customer’s intended ratings from social media posts. Benchmarking: An International Journal, 28(2): 438–464.  https://doi.org/10.1108/BIJ-01-2020-0004    [Google Scholar]
  50. Saad MM, Jamil N, and Hamzah R (2018). Evaluation of support vector machine and decision tree for emotion recognition of Malay folklores. Bulletin of Electrical Engineering and Informatics, 7(3): 479–486.  https://doi.org/10.11591/eei.v7i3.1279    [Google Scholar]
  51. Sailunaz K and Alhajj R (2019). Emotion and sentiment analysis from Twitter text. Journal of Computational Science, 36: 101003.  https://doi.org/10.1016/j.jocs.2019.05.009    [Google Scholar]
  52. Saputri MS, Mahendra R, and Adriani M (2018). Emotion classification on Indonesian Twitter dataset. In the International Conference on Asian Language Processing, IEEE, Bandung, Indonesia: 90-95.  https://doi.org/10.1109/IALP.2018.8629262    [Google Scholar]
  53. Sarakit P, Theeramunkong T, Haruechaiyasak C, and Okumura M (2015). Classifying emotion in Thai YouTube comments. In the 6th International Conference of Information and Communication Technology for Embedded Systems, IEEE, Hua Hin, Thailand: 1-5.  https://doi.org/10.1109/ICTEmSys.2015.7110808    [Google Scholar]
  54. Saravia E, Liu HCT, Huang YH, Wu J, and Chen YS (2018). CARER: Contextualized affect representations for emotion recognition. In the Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Brussels, Belgium: 3687-3697.  https://doi.org/10.18653/v1/D18-1404    [Google Scholar]
  55. Shameem M, Kumar RR, Kumar C, Chandra B, and Khan AA (2018). Prioritizing challenges of agile process in distributed software development environment using analytic hierarchy process. Journal of Software: Evolution and Process, 30(11): e1979.  https://doi.org/10.1002/smr.1979    [Google Scholar]
  56. Sintsova V, Musat C, and Pu P (2014). Semi-supervised method for multi-category emotion recognition in tweets. In the IEEE International Conference on Data Mining Workshop, IEEE, Shenzhen, China: 393-402.  https://doi.org/10.1109/ICDMW.2014.146    [Google Scholar]
  57. Sreeja PS and Mahalakshmi GS (2019). Emotion recognition in poetry using ensemble of classifiers. In the Next Generation Computing Technologies on Computational Intelligence: 4th International Conference, Springer Singapore, Dehradun, India: 77-91.  https://doi.org/10.1007/978-981-15-1718-1_7    [Google Scholar]
  58. Suhasini M and Srinivasu B (2020). Emotion detection framework for Twitter data using supervised classifiers. In the Data Engineering and Communication Technology: Proceedings of 3rd ICDECT-2K19, Springer Nature, Singapore, Singapore: 565-576.  https://doi.org/10.1007/978-981-15-1097-7_47    [Google Scholar]
  59. Tian F, Gao P, Li L, Zhang W, Liang H, Qian Y, and Zhao R (2014). Recognizing and regulating e-learners’ emotions based on interactive Chinese texts in e-learning systems. Knowledge-Based Systems, 55: 148-164.  https://doi.org/10.1016/j.knosys.2013.10.019    [Google Scholar]
  60. Tuhin RA, Paul BK, Nawrine F, Akter M, and Das AK (2019). An automated system of sentiment analysis from Bangla text using supervised learning techniques. In the IEEE 4th International Conference on Computer and Communication Systems, IEEE, Singapore, Singapore: 360-364.  https://doi.org/10.1109/CCOMS.2019.8821658    [Google Scholar]
  61. Xian G, Guo Q, Zhao Z, Luo Y, and Mei H (2023). Short text classification model based on DeBERTa-DPCNN. In the 4th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering, IEEE, Hangzhou, China: 56-59.  https://doi.org/10.1109/ICBAIE59714.2023.10281320    [Google Scholar]
  62. Yuan Z and Purver M (2015). Predicting emotion labels for Chinese microblog texts. In: Gaber M, Cocea M, Wiratunga N, Goker A (Eds.), Advances in social media analysis: 129-149. Springer, Cham, Switzerland.  https://doi.org/10.1007/978-3-319-18458-6_7    [Google Scholar]
  63. Zhang F, Xu H, Wang J, Sun X, and Deng J (2016). Grasp the implicit features: Hierarchical emotion classification based on topic model and SVM. In the International Joint Conference on Neural Networks, IEEE, Vancouver, Canada: 3592-3599.  https://doi.org/10.1109/IJCNN.2016.7727661    [Google Scholar]