The maximum amount of text a language model can process at once, measured in tokens.
The maximum amount of text a language model can process at once, measured in tokens. GPT-4 Turbo has a 128K context window; Claude can handle 200K+. Larger context windows let models work with longer documents but use more memory and compute. A key differentiator between models.
The basic unit of text that language models work with.
The neural network architecture behind virtually all modern AI language models.
An AI model that understands and generates human language.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.