Perbandingan Kinerja dan Efisiensi Full Fine-Tuning dan LoRA untuk Deteksi Email Phishing pada Model Transformer

Abstract

Serangan phishing merupakan ancaman keamanan siber yang terus berkembang dan semakin adaptif, sehingga pendekatan deteksi berbasis machine learning klasik menjadi kurang efektif. Model deep learning berbasis Transformer telah terbukti unggul dalam memahami semantik teks, namun penerapannya melalui skema full fine-tuning memerlukan sumber daya komputasi yang tinggi. Keterbatasan ini mendorong kebutuhan akan metode yang lebih efisien tanpa mengorbankan kinerja deteksi. Penelitian ini mengevaluasi efektivitas Parameter-Efficient Fine-Tuning (PEFT) menggunakan metode Low-Rank Adaptation (LoRA) untuk deteksi email phishing. Eksperimen dilakukan pada dataset publik yang terdiri dari 18.644 email dengan membandingkan lima arsitektur Transformer encoder, yaitu RoBERTa, BERT, ELECTRA, DeBERTa, dan DistilBERT. Evaluasi berfokus pada analisis trade-off antara kinerja klasifikasi, yang diukur menggunakan akurasi, presisi, recall, dan F1-score, serta efisiensi komputasi berdasarkan penggunaan VRAM dan waktu pelatihan. Hasil eksperimen menunjukkan bahwa LoRA mampu mempertahankan performa deteksi yang kompetitif dengan penurunan performa rata-rata kurang dari 1% dibandingkan full fine-tuning. BERT dengan full fine-tuning mencapai F1-score tertinggi sebesar 98,23%. Menariknya, pada DeBERTa, penerapan LoRA justru menghasilkan sedikit peningkatan performa hingga 98,13% dibandingkan versi full fine-tuning sebesar 98,02%, yang mengindikasikan efek regularisasi yang cukup efektif. Dari sisi efisiensi, LoRA mampu menurunkan konsumsi memori pada seluruh model, dengan penghematan tertinggi pada DistilBERT hingga 40%. Berdasarkan temuan ini, penggunaan full fine-tuning direkomendasikan jika prioritas utama adalah akurasi maksimal, sedangkan LoRA lebih sesuai untuk efisiensi memori.

Kata kunci: deteksi phishing, Transformer, Low-Rank Adaptation (LoRA), efisiensi komputasi, keamanan siber

 ----------------------------------------------------------------------

Comparison of Performance and Efficiency between Full Fine-Tuning and LoRA for Phishing Email Detection using Transformer Models

Phishing attacks represent an evolving and increasingly adaptive cybersecurity threat, rendering classical machine learning-based detection approaches less effective. Transformer-based deep learning models have demonstrated superiority in comprehending textual semantics, but their training via full fine-tuning schemes demands substantial computational resources. These limitations necessitate more efficient methods that do not compromise detection performance. This study evaluates the effectiveness of Parameter-Efficient Fine-Tuning (PEFT) using the Low-Rank Adaptation (LoRA) method for phishing email detection. Experiments were conducted on a public dataset comprising 18,644 emails, comparing five Transformer encoder architectures: RoBERTa, BERT, ELECTRA, DeBERTa and DistilBERT. The evaluation focuses on analyzing the trade-off between classification performance, measured using accuracy, precision, recall, & F1-score, and computational efficiency based on VRAM usage and training time. Experimental results demonstrate that LoRA is capable of maintaining competitive detection performance with an average performance degradation of less than 1% compared to full fine-tuning. The BERT model with full fine-tuning achieved the highest F1-score of 98.23%. Notably, in DeBERTa, the application of LoRA yielded a slight performance improvement to 98.13% compared to the full fine-tuning version (98.02%), indicating an effective regularization effect. In terms of efficiency, LoRA reduced memory consumption across all models, with the highest saving observed in DistilBERT, reaching up to 40%. Based on these findings, the use of full fine-tuning is recommended if the primary priority is maximum accuracy, whereas LoRA is more suitable for memory efficiency.

Keywords: phishing detection, Transformer, Low-Rank Adaptation (LoRA), computational efficiency, cybersecurity