A hyperparameter that controls how much the model's weights change in response to each update.
A hyperparameter that controls how much the model's weights change in response to each update. Too high and the model overshoots good solutions and diverges. Too low and training takes forever or gets stuck. Learning rate schedules that decrease over time are common practice.
The fundamental optimization algorithm used to train neural networks.
A setting you choose before training begins, as opposed to parameters the model learns during training.
The process of teaching an AI model by exposing it to data and adjusting its parameters to minimize errors.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.