US Patent Issued to Hewlett Packard Enterprise Development on July 1 for "Efficient key-value cache management for large language models" (California Inventors)

ALEXANDRIA, Va., July 3 -- United States Patent no. 12,346,252, issued on July 1, was assigned to Hewlett Packard Enterprise Development LP (Spring, Texas).

"Efficient key-value cache management for large language models" was invented by Aditya Dhakal (Santa Clarita, Calif.), Pedro H. R. Bruel (San Jose, Calif.), Gourav Rattihalli (Milpitas, Calif.), Sai Rahul Chalamalasetti (Newark, Calif.) and Dejan S. Milojicic (Palo Alto, Calif.).

According to the abstract* released by the U.S. Patent & Trademark Office: "A first NIC monitors a key-value cache associated with an LLM executed by a compute node that includes the first NIC and an accelerator. The key-value cache is stored in a memory associated with the accelerator. Responsive to detecti...