Prompting Kimi K3

Kimi K3 is 2.8 trillion parameters with 104 billion active per token, it went live on Moonshot’s hosted API on 16 July 2026, and its weights followed on 27 July. Prompting it is simpler than prompting K2 was, mostly because Moonshot removed the parameters people used to spend their time tuning.

Which parameters are fixed, and what happens if you send them?

Temperature is fixed at 1.0 and is not yours to set. So are top_p at 0.95, n at 1, and both presence_penalty and frequency_penalty at 0. Moonshot’s own parameter reference tells you to omit them from requests rather than pass the documented value.

This is a real break from K2, and any guide you have read about scaling Kimi’s temperature is now about the wrong model. K2-era models kept some latitude, and the Anthropic-compatible endpoint for K2 documented a temperature remap. None of that carries to K3, which replaces the whole axis with one field.

What does reasoning_effort do, and when should you move it?

It is a top-level request field taking low, high or max, and it defaults to max. Thinking cannot be switched off at any setting, so treat the choice as how long the model deliberates rather than whether it does.

The cost of changing it mid-session is the part worth planning around: switching reasoning_effort invalidates prefix-cache hits, so a conversation that relies on caching should pick one level and hold it. On a $3 input price where a hit costs $0.30, that is a tenfold difference on the repeated part of every request.

The tech report explains why the dial behaves as it does. Moonshot trained it with what it calls Reasoning Effort RL, a per-problem token budget with a reward penalty for exceeding a scaled threshold, and for agentic tasks the counted budget accumulates across the whole trajectory rather than resetting per turn.

How do you get a prefix cache hit?

Automatically, and there is no cache API to call. Moonshot’s wording is that context caching “is automatically enabled for all model requests” with no manual creation or management.

The eligibility rule is the thing to design for: a request can hit the prefix cache only when the previous request’s prompt exceeded 256 tokens. Below that, nothing was stored to hit. Moonshot’s recommended layout follows from it: put fixed large context such as knowledge documents at the beginning of the messages array, before the system message, then append the user’s question after it. Hits bill at $0.30 per million input tokens against $3.00 on a miss, and Moonshot reports lower first-token latency with them.

Its own guidance on when to reach for caching rather than retrieval is unusually concrete: prioritise caching for frequent queries against fixed content such as FAQs and document question-answering, and consider retrieval only if the content is extremely large.

Tools, JSON and the message you must not truncate

The tool surface is OpenAI-shaped: a tools array of {type: "function", function: {name, description, parameters}}, tool_calls on the assistant message, results returned per call. tool_choice accepts auto (the default), none, required and a named tool, and K3 is the only current Kimi model that supports required.

One documented incompatibility: forcing a specific named tool while thinking is enabled returns a 400. Since thinking cannot be disabled on K3, forcing a named tool is effectively unavailable; required is the setting to reach for instead.

The mistake that breaks multi-turn agent loops is smaller than it looks. Moonshot states it directly: “For multi-turn conversations and tool calls, add the complete assistant message returned by the API to the next request. Do not keep only content.” Strip an assistant message down to its text and you have thrown away the structure the next turn depends on.

For structured output there are two mechanisms. Loose JSON mode is response_format: {"type": "json_object"}. Strict schema constraint is response_format: {"type": "json_schema", "json_schema": {"name": ..., "strict": true, "schema": {...}}}, which constrains the final content field. Because response_format is appended after the input messages in the template, switching it per request does not destroy the prefix cache.

Prefill works, which is worth knowing given it does not on Anthropic’s current models: append a final assistant message carrying "partial": true and your prefix text, and the model continues from it. The API’s response does not include your prefix, so prepend it yourself when displaying the result.

What do self-hosters need that the model card does not spell out?

The chat template, because K3 does not ship a portable one. There is no chat_template.jinja in the Hugging Face repository. Rendering is implemented in Python: TikTokenTokenizer.apply_chat_template() in tokenization_kimi.py, which calls build_chat_segments() in encoding_k3.py. The format itself is a Moonshot-coined markup called XTML, described in the tech report’s appendix, and the tokenizer is tiktoken-based BPE with a 160K vocabulary, not SentencePiece.

The special tokens actually shipped in tokenizer_config.json include <|end_of_msg|>, [start_header_id], [end_header_id] and [EOT]. The tokenizer’s Python constructor also defines a fallback list containing <|im_end|>, <|im_user|> and similar; that list is not what the released model uses, and picking it up by mistake is the kind of error that produces a model which answers but never quite behaves.

Moonshot recommends vLLM, SGLang and KTransformers and links each project’s own recipe rather than publishing launch commands, GPU counts or tensor-parallel settings of its own. Quantisation is quantisation-aware training from the supervised fine-tuning stage onward, MXFP4 weights with MXFP8 activations; the tech report adds that only the routed expert weights are quantised, with attention projections and the rest left alone.

Read clause 2 before you resell it

The Kimi K3 licence is MIT-style with two commercial carve-outs. Model-as-a-service, defined as giving a third party access to inference or fine-tuning in a way that lets them exercise meaningful control, triggers a separate agreement above $20 million in trailing twelve-month revenue, aggregated with affiliates. Above 100 million monthly active users or $20 million in monthly revenue, “Kimi K3” must be displayed prominently in the product’s interface. Internal use is exempt, as is access through Moonshot’s own products and certified inference partners.

For most teams neither clause binds. For anyone putting K3 behind their own API as a product, both are the reason to read the licence rather than assume the MIT part covers it.