Speculative KV coding: losslessly compressing KV cache by up to ~4×

123 points

by kkm

3 days ago

20 comments