The process of taking a pre-trained model and continuing to train it on a smaller, specific dataset to adapt it for a particular task or domain.
The process of taking a pre-trained model and continuing to train it on a smaller, specific dataset to adapt it for a particular task or domain. Much cheaper and faster than training from scratch. LoRA and QLoRA are popular efficient fine-tuning methods that only update a small fraction of parameters.
The initial, expensive phase of training where a model learns general patterns from a massive dataset.
Low-Rank Adaptation.
Using knowledge learned from one task to improve performance on a different but related task.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.