What Is a Chat Template?

A chat template formats conversational messages into the structured input that a language model was trained to understand.

Definition

A chat template is a formatting rule that converts a conversation into the exact sequence of text or tokens expected by a conversational AI model. It usually defines how system instructions, user messages, assistant replies, tool outputs, and special control markers are arranged before the model processes them.

Chat templates belong to the interface layer between an application and a language model. They matter because a model trained with one conversational format may perform poorly when messages are presented in another, even when the visible wording is identical. A correct chat template helps the model recognize who said what, where a reply should begin, and which instructions have priority.

In One Sentence

A chat template formats conversational messages into the structured input that a language model was trained to understand.

Key Takeaways

• A chat template converts structured messages into a model-specific sequence of text or tokens.

• It identifies roles such as system, user, assistant, and tool.

• Different models may require different chat templates even when they receive the same visible conversation.

• An incorrect template can reduce answer quality, confuse message roles, or cause unwanted text generation.

• Chat templates affect formatting and interpretation but do not change the model’s trained knowledge.

Why Chat Templates Matter

Users often interact with AI through a simple chat interface. Behind that interface, however, the model may not receive a neatly separated list of messages. It usually receives one continuous sequence of tokens containing special markers that indicate where each message begins and ends.

The chat template creates that sequence.

Readers are likely to encounter chat templates when running open-weight language models, configuring inference software, using model-serving frameworks, building chat applications, or inspecting tokenizer settings. Hosted AI services often handle this process automatically, while local or custom deployments may require the developer to select or configure the correct template.

Understanding chat templates improves practical knowledge of AI because it explains why the same model can behave differently across applications. One interface may format messages correctly, while another may omit role markers, duplicate instructions, or place the assistant prompt in the wrong position.

In real-world use, these differences can affect instruction following, response style, tool use, safety behaviour, and whether the model continues the conversation correctly. A chat template is not glamorous, but neither is punctuation until somebody removes it.

How a Chat Template Works

A chat application usually stores a conversation as a list of structured messages. Each message commonly includes a role and some content.

For example, an application might represent a conversation conceptually like this:

• System: You are a concise assistant.

• User: Explain photosynthesis.

• Assistant: Photosynthesis is the process...

• User: Make that simpler.

The model may not accept this list directly. Instead, the messages must be combined into a single sequence using the markers and layout expected during training.

A simplified rendered version might resemble:

<system>
You are a concise assistant.
</system>
<user>
Explain photosynthesis.
</user>
<assistant>
Photosynthesis is the process...
</assistant>
<user>
Make that simpler.
</user>
<assistant>

The final assistant marker indicates that the model should now generate the next reply.

Actual templates vary considerably. Some use visible labels, while others use special tokens that do not appear as ordinary text. A model may expect dedicated beginning-of-message and end-of-message tokens, role names, separators, or generation markers.

The template may also control:

• Whether a system message is allowed

• Where system instructions appear

• How multiple user and assistant turns are separated

• Whether an end-of-sequence token is added

• How tool calls and tool results are represented

• Whether an assistant generation prompt is appended

• How empty or missing roles are handled

A tokenizer or model configuration often stores the chat template. The application passes the message list to a rendering function, which applies the template and produces the final model input.

Many templates are written using a small templating language. The template loops through messages, checks each role, inserts the required special tokens, and places message content in the expected positions.

For example, a template may contain logic roughly equivalent to:

For every message:
    add the token that begins a message
    add the message role
    add the message content
    add the token that ends a message

If generating a reply:
    add the marker for a new assistant message

This procedure may look simple, but small differences matter.

Suppose a model was trained to see each assistant reply followed by a special end marker. If an application omits that marker, the model may continue generating beyond the intended answer. If user and assistant labels are reversed, the model may imitate the user instead of responding to the user.

A chat template can also include tool-related roles. In an AI system that calls external functions, the conversation may include an assistant tool request followed by a tool result. The template must represent these messages in a format the model recognizes.

For example, the sequence may need to distinguish between:

• An ordinary assistant reply

• A request to call a calculator

• The calculator’s returned result

• The assistant’s final explanation

If the template flattens all of these into ordinary text, tool use may become unreliable.

Chat templates are especially important for models that were fine-tuned for conversation. A base language model may simply predict the next token from plain text. An instruction-tuned or chat-tuned model has usually learned a particular dialogue structure. Using that structure during inference helps reproduce the behaviour developed during training.

Common Misconceptions About Chat Templates

‘A chat template is the same as a prompt.’

A prompt is the content given to a model. A chat template is the rule used to arrange that content into the model’s expected conversational format. The template may contain fixed markers, but it is not the same as the user’s instructions.

‘All language models use the same chat template.’

They do not. Different model families may use different role markers, separators, special tokens, and system-message rules. Even related models can require slightly different formatting.

‘A chat template changes the model’s intelligence.’

A template does not alter the model’s parameters or add knowledge. It can, however, help or hinder access to the behaviour the model learned during training. Poor formatting can make a capable model appear less reliable.

‘Visible role labels are enough.’

Writing ‘User:’ and ‘Assistant:’ may work for some models, but others expect specific special tokens or formatting. Human-readable labels are not always equivalent to the model’s trained dialogue format.

‘Chat templates matter only for local models.’

Local deployments expose them more visibly, but hosted systems also use message formatting internally. The difference is that the service provider usually manages it for the user.

Comparing Chat Templates with Similar Concepts

Chat Template vs Prompt Template

A prompt template provides reusable wording or placeholders for a task, such as a template for summarization or classification.

A chat template operates at a lower level. It formats message roles and boundaries so the model can interpret the conversation correctly. A prompt template may be inserted inside a chat template.

Chat Template vs System Prompt

A system prompt contains instructions about the assistant’s role, behaviour, or constraints.

A chat template determines where and how that system prompt is placed relative to user and assistant messages. The system prompt is content; the chat template is structure.

Chat Template vs Tokenizer

A tokenizer converts text into tokens that the model can process.

A chat template first arranges messages into the expected conversational sequence. The tokenizer then converts that rendered sequence, including any special markers, into token identifiers. In practice, the two are often closely integrated.

Chat Template vs Conversation History

Conversation history is the collection of earlier messages in a dialogue.

The chat template determines how that history is serialized and presented to the model. The history supplies the content, while the template supplies the formatting rules.

Chat Template vs Model Configuration

Model configuration describes technical properties such as architecture, vocabulary size, context length, and special token identifiers.

A chat template may be stored within or alongside that configuration, but it serves the narrower purpose of formatting conversational input.

See Also

Large Language Model

A chat template exists to prepare conversational input for a language model. Understanding how large language models process token sequences provides the foundation for understanding why formatting matters.

Token

Chat templates ultimately produce sequences of tokens. Learning what tokens are helps explain why invisible control markers can influence model behaviour.

Tokenizer

The tokenizer converts the rendered chat template into the numerical token sequence consumed by the model. It is the most immediate technical component to study next.

System Prompt

System prompts are commonly inserted through chat templates. Exploring system prompts clarifies how high-level instructions are separated from ordinary user messages.

Prompt Template

Prompt templates standardize reusable task instructions, while chat templates standardize the surrounding conversational structure. Comparing the two prevents a common terminology mix-up.

Instruction Tuning

Instruction tuning teaches a model to follow requests presented in particular formats. Chat templates often reproduce those formats during inference.

Special Token

Special tokens mark roles, boundaries, sequence endings, and other control information. They are central to how many chat templates communicate structure to a model.

Context Window

The fully rendered chat template, including conversation history and control tokens, occupies part of the context window. Understanding this relationship helps explain why long conversations eventually reach input limits.

Inference

Chat templates are generally applied during inference, when a trained model receives messages and generates a response. Studying inference places the template within the wider model workflow.