Helicone

The practice of storing pre-computed Key-Value (KV) states of prompt tokens on model servers to eliminate redundant processing and drastically reduce latency and cost for repetitive context.