US Patent Issued to GOOGLE on March 25 for "Extreme language model compression with optimal sub-words and shared projections" (Washington, California, Pennsylvania Inventors)

ALEXANDRIA, Va., March 26 -- United States Patent no. 12,260,340, issued on March 25, was assigned to GOOGLE LLC (Mountain View, Calif.).

"Extreme language model compression with optimal sub-words and shared projections" was invented by Yang Song (Bellevue, Wash.), Raghav Gupta (Mountain View, Calif.), Dengyong Zhou (Redmond, Wash.) and Sanqiang Zhao (Pittsburgh).

According to the abstract* released by the U.S. Patent & Trademark Office: "Provided is a knowledge distillation technique for training a student language model that, relative to a larger teacher language model, has a significantly smaller vocabulary, lower embedding dimensions, and/or hidden state dimensions. Specifically, aspects of the present disclosure are directed to a dua...