Throughput

Throughput: The total amount of work a serving system completes per unit time, often measured in tokens or requests per second across all users.

Browse the full glossary →