Hy4 preview

What it is

Hy4 preview is Tencent’s new flagship language model, published on 27 August 2026 with its weights open from the first day under an Apache 2.0 licence. Tencent’s own model card introduces it as “a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team”, carrying 770 billion total parameters “of which 49B are activated per token”.

A mixture-of-experts model divides its parameters into many specialist blocks and fires only a few of them for each token, so the cost of serving it tracks the active count rather than the full total. Hy4 preview runs 78 layers. The first uses a conventional dense feed-forward block, and the other 77 replace it with a mixture of 256 routed experts plus one shared expert, with each token activating the top eight routed experts alongside the shared one. Bolted to that backbone is a separate multi-token-prediction layer of 10 billion parameters, 0.7 billion of them active, which exists to speed up speculative decoding. Context runs to one million tokens.

The rest of the specification is on the card and in the configuration files: hidden size 6144, 64 attention heads, an expert intermediate size of 2048 against 18432 for the dense block, four residual streams, a vocabulary of 120,832, and an architecture class Hugging Face records as HYV4ForCausalLM. Modality is text in, text out.

What “preview” covers, and what it does not

The label is part of the name, in the repository tencent/Hy4-preview and in Tencent’s own prose, so it is worth being exact about what is gated. The weights are not. Tencent published them on the day of release across Hugging Face, ModelScope, GitCode and its own CNB, in full precision and as a separate FP8 quantisation, with documented deployment on vLLM and SGLang and sections of the card devoted to finetuning and quantising the model yourself. Within a day it was on OpenRouter as a hosted endpoint.

What the label does flag is how much of the evidence is still unpublished. The benchmark figures on the card sit inside an image rather than a table, so nothing can be lifted from the primary source and re-checked, and Tencent devotes a section of the same card to the limitations it already knows about. No independent scoring existed at the end of August.

The architecture, with the borrowings named

The interesting paragraph on the card is the one about where the design came from. Tencent writes that “inspired by DeepSeek and GLM, the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse”, and that “the residual pathway uses iHC (identity Hyper-Connections) to expand inter-layer information flow”. The indexer that picks which earlier tokens each layer attends to is set at a top-k of 2048.

Both ideas are traceable. The card cites the DeepSeek Sparse Attention paper (arXiv 2512.02556) and the IndexCache paper (arXiv 2603.12201), and the shape of the design is familiar from the two labs named: DeepSeek V4 is built on compressed sparse attention and hyper-connections of its own, and Z.ai used Manifold-Constrained Hyper-Connections in the newly trained base behind GLM-5.3-Flash, published the day before Hy4 preview. A Chinese frontier lab crediting two Chinese rivals by name inside its own model card is a piece of publishing behaviour, and a useful one for anyone trying to trace which architectural ideas are actually spreading.

Who Tencent says it is for

Tencent describes going to its own staff for the training data, partnering with people inside the company including software engineers, game developers, finance analysts and security experts, and building training data around “the work they ship”. Four target areas follow from that: software engineering, office and analysis work covering documents, spreadsheets, presentations and financial models, game development from a single prompt to a playable prototype and on into game engines, and scientific research.

On the gain over the previous generation, Tencent claims “the largest generation-over-generation gain we’ve measured, and enough to put Hy4 preview at the open-source frontier”, scaled on model size, context length and training data. Because the supporting numbers are in an image, that sentence is currently the whole of the checkable claim.

Against the other open-weight flagships

Two readings come out of that. Hy4 preview is the second-smallest of the five by total parameters, above GLM-5.3’s 753 billion and far below the three giants, so Kimi K3 keeps the size record it set in July, and Hy4 preview is cheaper to serve per token than the two whose active counts are stated: 49 billion active parameters against Kimi K3’s 104 billion and Qwen3.8-Max’s 95 billion. And on licensing it lands where only DeepSeek has been. Kimi K3, Qwen3.8-Max and GLM-5.3 all ship under terms their makers wrote themselves, which makes open weight the accurate description rather than open source. Apache 2.0 is a standard licence with no revenue trigger and no bespoke clauses, so for this release open source is the correct word.

Where it runs, and what it costs

The two Hugging Face repositories are tencent/Hy4-preview and tencent/Hy4-preview-FP8. On 30 August, three days after release, Hugging Face’s own counters put them at 2,123 and 1,469 downloads. That is a very small number by the standards of the month, and the reason is the hardware bill: Z.ai’s 320-billion-parameter GLM-5.3-Flash had passed 346,000 downloads on the same day, and Alibaba’s Qwen3.8-Flash-Next, whose main model is 125 billion, had passed 121,000. A 770-billion-parameter model with a million-token window is not something most people pull down to try.

For everyone else there is the hosted route. OpenRouter listed tencent/hy4-preview on 28 August 2026 at $0.834 per million input tokens and $2.501 per million output, with the full 1,048,576-token window. Set against the rivals in the table, that undercuts Qwen3.8-Max at $2.00 and $6.00, and sits above DeepSeek V4 Pro’s off-peak $0.66 and $1.98 while coming in below its peak rate of $1.32 and $3.96.

Not a release: the Hy-MT2 listings

One thing to keep out of any list of August launches. Three Tencent translation models, Hy-MT2-1.8B, Hy-MT2-7B and Hy-MT2-30B-A3B, appeared on OpenRouter on 19 and 20 August 2026, which is easy to read as a launch. The weights are not new: all three Hugging Face repositories were created on 11 May 2026. What changed in August is where they are hosted.

What to watch

Three things. Whether the benchmark image on the card is followed by a technical report with numbers anyone can quote and reproduce. Whether independent evaluators pick the model up, which usually needs a stable hosted endpoint and has already begun via OpenRouter. And whether the licence survives: Apache 2.0 on a preview is a stronger commitment than most of this field has made, and the build that drops the word “preview” is the one that will show whether Tencent meant it.

For the wider Tencent family this sits inside, see Hunyuan; for the open-weight field it is measured against, Kimi K3, Qwen3.8-Max and DeepSeek V4 Pro.