The process of selecting the next token from the model's predicted probability distribution during text generation.
The process of selecting the next token from the model's predicted probability distribution during text generation. Different sampling strategies (greedy, temperature, top-k, top-p) produce different trade-offs between coherence and creativity. The choice dramatically affects output quality and character.
A parameter that controls the randomness of a language model's output.
A text generation method (also called nucleus sampling) that only considers tokens whose cumulative probability exceeds a threshold P.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
The research field focused on making sure AI systems do what humans actually want them to do.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.