Comparative Analysis of Hybrid CNN-ViT and CNN for Brain Tumor Classification
DOI:
https://doi.org/10.14421/jiska.5860Keywords:
Artificial Intelligence, Convolutional Neural Network, Machine Learning, Medical Image Analysis, Vision TransformerAbstract
The automated categorization of brain cancers from MRI is essential for improving diagnostic precision. Traditional Convolutional Neural Networks (CNNs) are proficient in local feature extraction but are constrained in their ability to capture long-range spatial relationships, hence impairing performance on intricate malignancies. We propose a hybrid parallel architecture that merges a CNN with a Vision Transformer (ViT) to combine local and global feature modeling. We assessed our dual-branch model in comparison to a conventional CNN baseline using a curated dataset of 15,000 MRI images categorized into three classes: glioma, meningioma, and pituitary. The hybrid model exhibited enhanced performance, attaining 98.40% accuracy and 0.0783 loss, in contrast to the baseline's 97.40% accuracy and 0.1187 loss. The substantial decrease in misclassifications was validated by additional metrics, such as enhanced recall for the meningioma category. The integration of local and global variables produces a more precise, stable, and generalizable classification framework, demonstrating significant potential as a basis for dependable AI-driven Clinical Decision Support Systems (CDSS) in neuroradiology.
References
Aggarwal, K., Manso Jimeno, M., Ravi, K. S., Gonzalez, G., & Geethanath, S. (2023). Developing and Deploying Deep Learning Models in Brain Magnetic Resonance Imaging: A Review. NMR in Biomedicine, 36(12), Article ID: e5014. https://doi.org/10.1002/nbm.5014
Alanazi, M. F., Ali, M. U., Hussain, S. J., Zafar, A., Mohatram, M., Irfan, M., AlRuwaili, R., Alruwaili, M., Ali, N. H., & Albarrak, A. M. (2022). Brain Tumor/Mass Classification Framework Using Magnetic-Resonance-Imaging-Based Isolated and Developed Transfer Deep-Learning Model. Sensors, 22(1), Article ID: 372. https://doi.org/10.3390/s22010372
Alayón, S., Hernández, J., Fumero, F. J., Sigut, J. F., & Díaz-Alemán, T. (2023). Comparison of the Performance of Convolutional Neural Networks and Vision Transformer-Based Systems for Automated Glaucoma Detection with Eye Fundus Images. Applied Sciences, 13(23), Article ID: 12722. https://doi.org/10.3390/app132312722
Ali, R. R., Yaacob, N. M., Alqaryouti, M. H., Sadeq, A. E., Doheir, M., Iqtait, M., Rachmawanto, E. H., Sari, C. A., & Yaacob, S. S. (2025). Learning Architecture for Brain Tumor Classification Based on Deep Convolutional Neural Network: Classic and ResNet50. Diagnostics, 15(5), Article ID: 624. https://doi.org/10.3390/diagnostics15050624
Avcı, D. (2025). A New Pes Planus Automatic Diagnosis Method: ViT-OELM Hybrid Modeling. Diagnostics, 15(7), Article ID: 867. https://doi.org/10.3390/diagnostics15070867
Balamurugan, T., & Gnanamanoharan, E. (2023). Brain Tumor Segmentation and Classification Using Hybrid Deep CNN with LuNetClassifier. Neural Computing and Applications, 35(6), 4739–4753. https://doi.org/10.1007/s00521-022-07934-7
Bukhari, M. T. (2024). Efficacy of Lightweight Vision Transformers in Diagnosis of Pneumonia. International Journal of Clinical Nephrology, 3(1). https://doi.org/10.37579/2834-5142/020
Checcucci, E., Piazzolla, P., Marullo, G., Innocente, C., Salerno, F., Ulrich, L., Moos, S., Quarà, A., Volpi, G., Amparore, D., Piramide, F., Turcan, A., Garzena, V., Garino, D., De Cillis, S., Sica, M., Verri, P., Piana, A., Castellino, L., … Porpiglia, F. (2023). Development of Bleeding Artificial Intelligence Detector (BLAIR) System for Robotic Radical Prostatectomy. Journal of Clinical Medicine, 12(23), Article ID: 7355. https://doi.org/10.3390/jcm12237355
Chibuike, O., & Yang, X. (2024). Convolutional Neural Network–Vision Transformer Architecture with Gated Control Mechanism and Multi-Scale Fusion for Enhanced Pulmonary Disease Classification. Diagnostics, 14(24), Article ID: 2790. https://doi.org/10.3390/diagnostics14242790
Dai, Y., Lian, C., Zhang, Z., Gao, J., Lin, F., Li, Z., Wang, Q., Chu, T., Aishanjiang, D., Chen, M., Wang, X., Cheng, G., Huang, R., Dong, J., Zhang, H., & Mao, N. (2025). Development and Validation of a Deep Learning System to Differentiate HER2-Zero, HER2-Low, and HER2-Positive Breast Cancer Based on Dynamic Contrast-Enhanced MRI. Journal of Magnetic Resonance Imaging, 61(5), 2212–2220. https://doi.org/10.1002/jmri.29670
Deng, J., Dong, W., Socher, R., Li, L.-J., Kai Li, & Li Fei-Fei. (2009). ImageNet: A Large-Scale Hierarchical Image Database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Dragan, P., Merski, M., Wiśniewski, S., Sanmukh, S. G., & Latek, D. (2023). Chemokine Receptors—Structure-Based Virtual Screening Assisted by Machine Learning. Pharmaceutics, 15(2), Article ID: 516. https://doi.org/10.3390/pharmaceutics15020516
Emara, H. M., El-Shafai, W., Soliman, N. F., Algarni, A. D., El-Samie, F. E. A., & Mahmoud, A. A. (2025). A Unified Vision Transformer and Convolutional Neural Network Framework for Multi-Domain Cancer Classification. In Research Square. https://doi.org/10.21203/rs.3.rs-6633290/v1
Fujima, N., Kamagata, K., Ueda, D., Fujita, S., Fushimi, Y., Yanagawa, M., Ito, R., Tsuboyama, T., Kawamura, M., Nakaura, T., Yamada, A., Nozaki, T., Fujioka, T., Matsui, Y., Hirata, K., Tatsugami, F., & Naganawa, S. (2023a). Current State of Artificial Intelligence in Clinical Applications for Head and Neck MR Imaging. Magnetic Resonance in Medical Sciences, 22(4), 401–414. https://doi.org/10.2463/mrms.rev.2023-0047
Fujima, N., Kamagata, K., Ueda, D., Fujita, S., Fushimi, Y., Yanagawa, M., Ito, R., Tsuboyama, T., Kawamura, M., Nakaura, T., Yamada, A., Nozaki, T., Fujioka, T., Matsui, Y., Hirata, K., Tatsugami, F., & Naganawa, S. (2023b). Current State of Artificial Intelligence in Clinical Applications for Head and Neck MR Imaging. Magnetic Resonance in Medical Sciences, 22(4), 401–414. https://doi.org/10.2463/mrms.rev.2023-0047
Goyal, A. K., Pandey, M., Singh, D. P., Choudhary, J., & Haripriya, R. (2025). FedContrast: A Contrastive Learning Framework to Mitigate Client Drift Under Statistical Heterogeneity in Federated Multi-Class Soil Classification. In Research Square. https://doi.org/10.21203/rs.3.rs-6774369/v1
Hasan, M. A., Haque, F., Sarker, H., Abdullah, R., Roy, T., Taaha, N., Arafat, Y., Patwary, A. K., Ahsan, M., & Haider, J. (2025). Mulberry Leaf Disease Detection by CNN-ViT with XAI Integration. PLOS One, 20(6), Article ID: e0325188. https://doi.org/10.1371/journal.pone.0325188
Hossain, Z., Hossain, M. E., Ahmed, N., Kabir, M. F., & Hossain, I. S. (2025). Evaluating the Performance of Vision Transformers and Convolutional Neural Networks for Hostile Image Detection. Indonesian Journal of Advanced Research, 4(1), 111–130. https://doi.org/10.55927/ijar.v4i1.13681
Ishrak, M. F., Rahman, M. M., Joy, M. I. K., Tamuly, A., Akter, S., Tanim, D. M., Jawar, S., Ahmed, N., & Rahman, M. S. (2025). Vision Transformer Embedded Feature Fusion Model with Pre-Trained Transformers for Keratoconus Disease Classification. Emerging Science Journal, 9(2), 1037–1075. https://doi.org/10.28991/ESJ-2025-09-02-027
Jamali, A., Roy, S. K., Hong, D., Lu, B., & Ghamisi, P. (2024). How to Learn More? Exploring Kolmogorov–Arnold Networks for Hyperspectral Image Classification. Remote Sensing, 16(21), Article ID: 4015. https://doi.org/10.3390/rs16214015
Keng, M., & Merz, K. M. (2024). Eliminating the Deadwood: A Machine Learning Model for CCS Knowledge-Based Conformational Focusing for Lipids. Journal of Chemical Information and Modeling, 64(20), 7864–7872. https://doi.org/10.1021/acs.jcim.4c01051
Khatun, R., Chatterjee, S., Bert, C., Wadepohl, M., Ott, O. J., Semrau, S., Fietkau, R., Nürnberger, A., Gaipl, U. S., & Frey, B. (2025). Complex-Valued Neural Networks to Speed-Up MR Thermometry During Hyperthermia Using Fourier PD and PDUNet. Scientific Reports, 15(1), Article ID: 11765. https://doi.org/10.1038/s41598-025-96071-x
Kim, J. W., Khan, A. U., & Banerjee, I. (2025). Systematic Review of Hybrid Vision Transformer Architectures for Radiological Image Analysis. Journal of Imaging Informatics in Medicine, 38(5), 3248–3262. https://doi.org/10.1007/s10278-024-01322-4
Krichen, M. (2023). Convolutional Neural Networks: A Survey. Computers, 12(8), Article ID: 151. https://doi.org/10.3390/computers12080151
Liu, S., Yue, W., Guo, Z., & Wang, L. (2024). Multi-Branch CNN and Grouping Cascade Attention for Medical Image Classification. Scientific Reports, 14(1), Article ID: 15013. https://doi.org/10.1038/s41598-024-64982-w
Liu, Z., Tong, L., Chen, L., Jiang, Z., Zhou, F., Zhang, Q., Zhang, X., Jin, Y., & Zhou, H. (2023). Deep Learning Based Brain Tumor Segmentation: A Survey. Complex & Intelligent Systems, 9(1), 1001–1026. https://doi.org/10.1007/s40747-022-00815-5
Ma, W., Chen, R., Zhang, K., Wu, S., & Ding, S. (2025). Instruct Where the Model Fails: Generative Data Augmentation via Guided Self-Contrastive Fine-Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(6), 5991–5999. https://doi.org/10.1609/aaai.v39i6.32640
Murugesan, G., Nagendran, P., & Natarajan, J. (2025). Advancing Brain Tumor Diagnosis: Deep Siamese Convolutional Neural Network as a Superior Model for MRI Classification. Brain‐X, 3(2). https://doi.org/10.1002/brx2.70028
Naren, O. S. (2024). Multi Cancer Dataset. Kaggle. https://doi.org/10.34740/KAGGLE/DSV/3415848
Omer, A. A. M. (2024). Image Classification Based on Vision Transformer. Journal of Computer and Communications, 12(04), 49–59. https://doi.org/10.4236/jcc.2024.124005
Park, Saeran, Ayana, G., Wako, B. D., Jeong, K. C., Yoon, S., & Choe, S. (2025). Vision Transformers for Low-Quality Histopathological Images: A Case Study on Squamous Cell Carcinoma Margin Classification. Diagnostics, 15(3), 260. https://doi.org/10.3390/diagnostics15030260
Park, So-yun, Ayana, G., Wako, B. D., Jeong, K. C., Yoon, S.-D., & Choe, S. (2025). Vision Transformers for Low-Quality Histopathological Images: A Case Study on Squamous Cell Carcinoma Margin Classification. Diagnostics, 15(3), Article ID: 260. https://doi.org/10.3390/diagnostics15030260
Parulian, O. S. (2025). Efficient Design and Compression of CNN Models for Rapid Character Recognition. Jurnal Ilmu Komputer dan Informasi, 18(1), 127–140. https://doi.org/10.21609/jiki.v18i1.1443
Pumperla, M., & Cahall, D. (2022). Elephas: Distributed Deep Learning with Keras & Spark. Journal of Open Source Software, 7(80), Article ID: 4073. https://doi.org/10.21105/joss.04073
Ruthven, M., Peplinski, A. M., Adams, D. M., King, A. P., & Miquel, M. E. (2023). Real-Time Speech MRI Datasets with Corresponding Articulator Ground-Truth Segmentations. Scientific Data, 10(1), Article ID: 860. https://doi.org/10.1038/s41597-023-02766-z
Senan, E. M., Jadhav, M. E., Rassem, T. H., Aljaloud, A. S., Mohammed, B. A., & Al-Mekhlafi, Z. G. (2022). Early Diagnosis of Brain Tumour MRI Images Using Hybrid Techniques between Deep and Machine Learning. Computational and Mathematical Methods in Medicine, 2022, 1–17. https://doi.org/10.1155/2022/8330833
Tabassum, I., & Nunavath, V. (2024). A Hybrid Deep Learning Approach for Multi-Class Cyberbullying Classification Using Multi-Modal Social Media Data. Applied Sciences, 14(24), Article ID: 12007. https://doi.org/10.3390/app142412007
Touvron, H., Cord, M., & Jégou, H. (2022). DeiT III: Revenge of the ViT. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, & T. Hassner (Eds.), Computer Vision – ECCV 2022 (pp. 516–533). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-20053-3_30
Tummala, S., Kadry, S., Bukhari, S. A. C., & Rauf, H. T. (2022). Classification of Brain Tumor from Magnetic Resonance Imaging Using Vision Transformers Ensembling. Current Oncology, 29(10), 7498–7511. https://doi.org/10.3390/curroncol29100590
Ullah, Z., Odeh, A., Khattak, I., & Al Hasan, M. (2023). Enhancement of Pre-Trained Deep Learning Models to Improve Brain Tumor Classification. Informatica, 47(6), 165. https://doi.org/10.31449/inf.v47i6.4645
Wibowo, F. A., Yulianto, T., Malun, N. O., Rionaldy, R., Yasin, V., & Siagian, R. C. (2025). Application of Machine Learning Methods for Classification of Gamma and Hadron Signals in High Energy Particle Detection. Jurnal Ilmu Komputer dan Informasi, 18(2), 181–205. https://doi.org/10.21609/jiki.v18i2.1489
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020). Visual Transformers: Token-based Image Representation and Processing for Computer Vision. http://arxiv.org/abs/2006.03677
Xie, X. (2023). Deep Learning-Based Image Classification of MRI Brain Image. Theoretical and Natural Science, 27(1), 46–51. https://doi.org/10.54254/2753-8818/27/20240670
Yohannes, R., & Al Rivan, M. E. (2022). Klasifikasi Jenis Kanker Kulit Menggunakan CNN-SVM. Jurnal Algoritme, 2(2), 133–144. https://doi.org/10.35957/algoritme.v2i2.2363
Yu, Z., Liu, F., & Li, Y. (2024). scTCA: A Hybrid Transformer-CNN Architecture for Imputation and Denoising of scDNA-seq Data. Briefings in Bioinformatics, 25(6), Article ID: bbae577. https://doi.org/10.1093/bib/bbae577
Zhang, Y., Li, Z., Nan, N., & Wang, X. (2023). TranSegNet: Hybrid CNN-Vision Transformers Encoder for Retina Segmentation of Optical Coherence Tomography. Life, 13(4), Article ID: 976. https://doi.org/10.3390/life13040976
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ahmad Fauzi, Achmad Lutfi Fuadi, Agus Heri Yunial
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish with this journal agree to the following terms as stated in http://creativecommons.org/licenses/by-nc/4.0
a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
