Context window

AI basics

Why it matters

The context window is the model's working memory for one conversation or task. Everything you send, everything it has said back and any documents you attach have to fit inside it; past that, older parts fall out or the request is refused.

A bigger window lets a model read a long contract, a codebase or a book in one go. Companies quote it in tokens, so a window of a million tokens is roughly several novels' worth of text, though cost and speed usually grow with how much of it you use.

Related terms

Token

A chunk of text, often part of a word, that models read and write. API prices are quoted per million tokens.

Large language model (LLM)

An AI model trained on huge amounts of text to predict the next word, which lets it answer questions, write and code. Grok is a family of LLMs.

Inference

Inference is running an already-trained AI model to produce an answer, such as a chatbot reply. It is separate from training, which builds the model, and it runs for every user request.