Volume 12, Issue 10 (October 2025), Pages: 36-44

----------------------------------------------

 Original Research Paper

Recealer: A malware detection method based on machine learning and deep learning models

 Author(s): 

 Asia Othman Aljahdali *, Elaf Maqadmi, Atouf Ghabashi, Deem Alsuoilme, Bayader Alluhaybi, Edmy Alboqami

 Affiliation(s):

 College of Computer Science and Engineering, University of Jeddah, Jeddah, Saudi Arabia

 Full text

    Full Text - PDF

 * Corresponding Author. 

   Corresponding author's ORCID profile:  https://orcid.org/0000-0002-9013-9465

 Digital Object Identifier (DOI)

  https://doi.org/10.21833/ijaas.2025.10.005

 Abstract

Raccoon Stealer malware is difficult to detect as it actively evades traditional methods. This study proposes Recealer, a hybrid detection model that integrates machine learning (ML) and deep learning (DL). The model applies static and dynamic analysis to extract features from sample files, which are first classified using an ML algorithm. Files with uncertain classification results are then transformed into grayscale images and analyzed by a convolutional neural network for improved precision. Experimental results demonstrate that Recealer achieves 94% overall detection accuracy, with the random forest algorithm attaining 97.53% in the ML stage and the DL stage reaching 95%. These findings indicate that Recealer is both efficient and reliable for detecting Raccoon Stealer malware.

 © 2025 The Authors. Published by IASE.

 This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/).

 Keywords

 Malware detection, Machine learning, Deep learning, Hybrid model, Feature extraction

 Article history

 Received 22 April 2025, Received in revised form 26 August 2025, Accepted 3 September 2025

 Data availability

The datasets generated and analyzed during the current study are available in the GitHub repository, https://github.com/ElafFayk/RecealerMalwareDetection. 

 Acknowledgment

No Acknowledgment. 

 Compliance with ethical standards

 Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

 Citation:

 Aljahdali AO, Maqadmi E, Ghabashi A, Alsuoilme D, Alluhaybi B, and Alboqami E (2025). Recealer: A malware detection method based on machine learning and deep learning models. International Journal of Advanced and Applied Sciences, 12(10): 36-44

  Permanent Link to this page

 Figures

  Fig. 1   Fig. 2  

 Tables

  Table 1  Table 2  Table 3  Table 4  Table 5

----------------------------------------------   

 References (15)

1. Alam S, Horspool RN, and Traore I (2013). MAIL: Malware Analysis Intermediate Language: A step towards automating and optimizing malware detection. In the Proceedings of the 6th International Conference on Security of Information and Networks, ACM, Aksaray, Turkey: 233-240.  https://doi.org/10.1145/2523514.2527006    [Google Scholar]
2. Bermejo Higuera J, Abad Aramburu C, Bermejo Higuera JR, Sicilia Urban MA, and Sicilia Montalvo JA (2020). Systematic approach to malware analysis (SAMA). Applied Sciences, 10(4): 1360.  https://doi.org/10.3390/app10041360    [Google Scholar]
3. Castiglioni I, Rundo L, Codari M et al. (2021). AI applications to medical images: From machine learning to deep learning. Physica Medica, 83: 9-24.  https://doi.org/10.1016/j.ejmp.2021.02.006    [Google Scholar] PMid:33662856
4. Chen A, Chen H, Xu L, Xie H, Qiao H, Lin Q, and Cai K (2020). A deep learning CNN architecture applied in smart near-infrared analysis of water pollution for agricultural irrigation resources. Agricultural Water Management, 240: 106303.  https://doi.org/10.1016/j.agwat.2020.106303    [Google Scholar]
5. Elsayed MS, Le-Khac NA, Dev S, and Jurcut AD (2020). DDoSNet: A deep-learning model for detecting network attacks. In the IEEE 21st International Symposium on a World of Wireless, Mobile and Multimedia Networks, IEEE, Cork, Ireland: 391-396.  https://doi.org/10.1109/WoWMoM49955.2020.00072    [Google Scholar]
6. Hussain A, Asif M, Ahmad MB, Mahmood T, and Raza MA (2022). Malware detection using machine learning algorithms for Windows platform. In: Ullah A, Anwar S, Rocha Á, and Gill S (Eds.), Proceedings of International Conference on Information Technology and Applications. Lecture Notes in Networks and Systems, 350: 619–632. Springer, Singapore, Singapore.  https://doi.org/10.1007/978-981-16-7618-5_53    [Google Scholar]
7. Nurmi J, Niemelä M, and Brumley BB (2023). Malware Finances and operations: A data-driven study of the value chain for infections and compromised access. In the Proceedings of the 18th International Conference on Availability, Reliability and Security, ACM, Benevento, Italy: 1-12.  https://doi.org/10.1145/3600160.3605047    [Google Scholar]
8. Pant D and Bista R (2021). Image-based malware classification using deep convolutional neural network and transfer learning. In the Proceedings of the 3rd International Conference on Advanced Information Science and System, ACM, Sanya, China: 1-6.  https://doi.org/10.1145/3503047.3503081    [Google Scholar] PMid:33929876
9. Prachi, Dabas N, and Sharma P (2023). MalAnalyser: An effective and efficient Windows malware detection method based on API call sequences. Expert Systems with Applications, 230: 120756.  https://doi.org/10.1016/j.eswa.2023.120756    [Google Scholar]
10. Rabadi D and Teo SG (2020). Advanced Windows methods on malware detection and classification. In the Proceedings of the 36th Annual Computer Security Applications Conference, ACM, Austin, USA: 54-68.  https://doi.org/10.1145/3427228.3427242    [Google Scholar]
11. Saeed MAH (2020). Malware in computer systems: Problems and solutions. International Journal on Informatics for Development, 9(1): 1-8.  https://doi.org/10.14421/ijid.2020.09101    [Google Scholar]
12. Singh J and Singh J (2020). Detection of malicious software by analyzing the behavioral artifacts using machine learning algorithms. Information and Software Technology, 121: 106273.  https://doi.org/10.1016/j.infsof.2020.106273    [Google Scholar]
13. Sreekumari P (2020). Malware detection techniques based on deep learning. In the IEEE 6th Intl Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS), IEEE, Baltimore, USA: 65-70.  https://doi.org/10.1109/BigDataSecurity-HPSC-IDS49724.2020.00023    [Google Scholar]
14. Sun R, Yuan X, He P, Zhu Q, Chen A, Gregio A, Oliveira D, and Li X (2022). Learning fast and slow: Propedeutica for real-time malware detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6): 2518-2529.  https://doi.org/10.1109/TNNLS.2021.3121248    [Google Scholar] PMid:34723811
15. Yucel C, Lockett A, Chalkias K, Mallis D, and Katos V (2021). MAIT: Malware analysis and intelligence tool. Information & Security, 50(1): 49-65.  https://doi.org/10.11610/isij.5024    [Google Scholar]