Context caching
AIContext caching: Storing the processed representation of a repeated prompt prefix so later requests reuse it, cutting cost and the time to first token.
Context caching: Storing the processed representation of a repeated prompt prefix so later requests reuse it, cutting cost and the time to first token.