Fine-tuning (deep learning)

From Systems analysis Wiki
Jump to navigation Jump to search

Fine-tuning is a transfer learning technique in machine learning where the parameters of a pre-trained model are adapted to a new, specific task. Instead of training a model from scratch, which requires vast amounts of data and computational resources, fine-tuning leverages the knowledge already encoded in the model's weights and "tunes" them for a specific purpose.

This approach has become the de-facto standard in deep learning, especially when working with large language models (LLMs) and computer vision models.

Concept

The fine-tuning process can be divided into two main stages:

1. Pre-training: A model (e.g., BERT or GPT) is trained on a very large and general dataset (e.g., the entire internet) using a self-supervised task (e.g., predicting the next word). During this stage, the model learns general patterns, syntax, semantics, and world knowledge.

2. Fine-tuning: The pre-trained model is taken as a starting point, and its weights are adjusted on a small, labeled dataset specific to the target task.

The key idea is that the knowledge acquired during pre-training is general-purpose and can be successfully transferred to solve many other, more specialized tasks.

The Fine-Tuning Process

A typical fine-tuning process includes the following steps:

1. Select a Pre-trained Model: A model is chosen whose foundational capabilities are suitable for the target task (e.g., BERT for text understanding tasks, GPT for text generation).

2. Adapt the Architecture: A new "head" layer, specific to the target task, is added to the pre-trained model. For example:

  • For text classification, a simple fully-connected layer with a softmax function is added.
  • For named entity recognition (NER), a classifier is added to the output of each token.

3. Train on the Target Dataset: The entire model (or part of it) is trained on the new, labeled dataset. During this step, the model's weights, including those of the pre-trained layers, are updated via gradient descent to minimize the loss function on the new task. 4. Use a Lower Learning Rate: Fine-tuning typically uses a significantly lower learning rate than pre-training. This is necessary to avoid drastically altering the useful knowledge already encoded in the model's weights, and instead to adjust them carefully.

Types of Fine-Tuning

Full Fine-Tuning

  • Principle: All parameters of the pre-trained model are updated along with the new "head" layer.
  • Advantages: Potentially offers the best performance, as the entire model adapts to the new task.
  • Disadvantages: Requires significant computational resources and memory, as gradients must be stored and updated for all parameters. There is a risk of catastrophic forgetting, where the model "forgets" the general knowledge acquired during pre-training.

Parameter-Efficient Fine-Tuning (PEFT)

This is a family of methods aimed at reducing the computational costs of fine-tuning. The main idea is to freeze most of the pre-trained model's parameters and train only a small number of new or selected existing parameters.

  • Examples of PEFT methods:
    • Adapters: Small, additional "adapter" layers are inserted into the Transformer architecture, and only these layers are trained.
    • LoRA (Low-Rank Adaptation): Instead of updating the full weight matrices, LoRA trains their low-rank updates. This can reduce the number of trainable parameters by thousands of times.
    • Prompt Tuning: Trainable "prompt" vectors are added to the input data and are tuned to solve the task, while the model itself remains frozen.
  • Advantages of PEFT:
    • Efficiency: Significantly reduces memory and computational requirements.
    • Modularity: Allows a single pre-trained model to be easily adapted for multiple tasks by storing only a small set of adapted weights for each.

Instruction Tuning

This is a specific type of fine-tuning aimed at improving an LLM's ability to follow natural language instructions.

  • How it works: The model is fine-tuned on a dataset consisting of "instruction-desired output" pairs.
  • Goal: To improve the model's ability to generalize to new, unseen tasks that can be described as instructions. Models like InstructGPT and FLAN-T5 are prominent examples of this approach.

See also

Bibliography

  • Howard, J.; Ruder, S. (2018). Universal Language Model Fine-tuning for Text Classification. arXiv:1801.06146.
  • Houlsby, N. et al. (2019). Parameter-Efficient Transfer Learning for NLP. arXiv:1902.00751.
  • Pfeiffer, J. et al. (2020). AdapterFusion: Non-Destructive Task Composition for Transfer Learning. arXiv:2005.00247.
  • Hu, E. J. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
  • Lester, B.; Al-Rfou, R.; Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv:2104.08691.
  • Ben Zaken, A.; Goldberg, Y.; Ravfogel, S. (2022). BitFit: Simple Parameter-Efficient Fine-Tuning for Transformer-based Masked Language-Models. ACL 2022.
  • Ouyang, L. et al. (2022). Training Language Models to Follow Instructions with Human Feedback. arXiv:2203.02155.
  • Han, Z. et al. (2024). Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. arXiv:2403.14608.
  • Bian, J. et al. (2025). A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning. arXiv:2504.21099.
  • Li, X. et al. (2025). Revisiting Fine-Tuning: A Survey of Parameter-Efficient Techniques and Future Directions. Preprints.org.
Retrieved from "https://systems-analysis.info/eng/index.php?title=Fine-tuning_(deep_learning)&oldid=147"