ALEXANDRIA, Va., July 30 -- United States Patent no. 12,373,427, issued on July 29, was assigned to Microsoft Technology Licensing LLC (Redmond, Wash.).
"Dynamic batching of prompt queries" was invented by Marko Radmilac (Bellevue, Wash.), Andrew James Wald (Redmond, Wash.) and Joshua Bryan Wyman Clemons (Seattle).
According to the abstract* released by the U.S. Patent & Trademark Office: "A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for ...