New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
MarketWatch

XConn Technologies and MemVerge Demonstrate CXL Memory Pool for KV Cache using NVIDIA Dynamo for breakthrough AI workload performance at 2025 OCP Global Summit

The MarketWatch News Department was not involved in the creation of this content. XConn Technologies and MemVerge Demonstrate CXL Memory Pool for KV Cache using NVIDIA Dynamo for breakthrough AI ...
Semiconductor Engineering

Dynamic KV Cache Scheduling in Heterogeneous Memory Systems for LLM Inference (Rensselaer Polytechnic Institute, IBM)

A new technical paper titled “Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System” was published by researchers at Rensselaer Polytechnic Institute and IBM. “Large ...
Semiconductor Engineering

AI Inference Needs A Mix-And-Match Memory Strategy

Interactive LLMs (chat, copilots, agents) with strict latency targets Long‑context reasoning (codebases, research, video) with massive KV (key value) cache footprints Ranking and recommendation models ...