Speculative decoding
AISpeculative decoding: A speed-up where a small draft model proposes several tokens ahead and the main model verifies them in one pass, accepting those it agrees with.
Speculative decoding: A speed-up where a small draft model proposes several tokens ahead and the main model verifies them in one pass, accepting those it agrees with.