Cache hit rate (prompt)
AICache hit rate (prompt): The share of input tokens served from a provider's prompt cache, which decides much of the real cost of a repetitive workload.
Cache hit rate (prompt): The share of input tokens served from a provider's prompt cache, which decides much of the real cost of a repetitive workload.