What is Tokens per second (TPS)?

Tokens per second (TPS)

Tokens per second (TPS): A measure of generation speed counting how many tokens a model produces each second, commonly used to compare serving performance.

Related terms

Completion tokens
Floating-point operations per second (FLOP/s)
Prompt tokens
Special tokens
Thinking tokens
Circuit Layer Operations Per Second (CLOPS)
Redefinition of the second
Second quantisation

Browse the full glossary →