Evaluating Lightweight and Full Fine-Tuning Strategies Against Classical Machine Learning for Protein Function Prediction
Motivation Protein language models (PLMs) have emerged as powerful tools for sequence-based prediction of protein function, yet systematic benchmarks comparing frozen embeddings, fine-tuning strategies like Low-Rank Adaptation (LoRA) and classical machine learning (ML) remain limited. We benchmarked four ML strategies: ML using amino acid descriptors (SL-AAFeat), ML using frozen embeddings from 20 PLMs across various pooling strategies (SL-Embed), full model fine-tuning (FT-Full) and LoRA-based fine-tuning (FT-LoRA). Performance was evaluated on the in-house VHH phage display dataset (VHH) for binding affinity prediction and the TAPE fluorescence dataset (FLS and FLS10) for mutational effect prediction. Results Model performance depended strongly on the dataset and adaptation strategy. Max pooling consistently improved embedding-based models, while amino acid descriptors remained competitive under specific datasets and resource constraints. Fine-tuning generally provided the highest predictive performance, but the advantage is not universal. Hyperparameter optimization significantly enhanced FT-LoRA, enabling it to outperform FT-Full on the VHH dataset with less than 10% model parameter adaptation. In contrast, FT-Full achieved the best performance on FLS and FLS10. Several medium-sized PLMs performed comparably to larger models, highlighting favorable performance-efficiency trade-offs. Overall, this paper presents a thorough review of PLM utilization strategies and practical recommendations for selecting suitable strategies based on dataset characteristics and available computational resources. Availability The source code used in this manuscript is available in a Zenodo repository at https://doi.org/10.5281/zenodo.21466255.