US Patent Issued to GOOGLE on Dec. 30 for "Machine-learned attention models featuring echo-attention layers" (Dutch, Singaporean, American Inventors)

ALEXANDRIA, Va., Dec. 31 -- United States Patent no. 12,511,521, issued on Dec. 30, was assigned to GOOGLE LLC (Mountain View, Calif.).

"Machine-learned attention models featuring echo-attention layers" was invented by Yi Tay (Singapore), Donald Arthur Metzler Jr. (Sunnyvale, Calif.), Dara Bahri (Lafayette, Calif.) and Mostafa Dehghani (Amsterdam).

According to the abstract* released by the U.S. Patent & Trademark Office: "The present disclosure provides echo-attention layers, a new efficient method for increasing the expressiveness of self-attention layers without incurring significant parameter or training time costs. One intuition behind the proposed method is to learn to echo, i.e., attend once and then get N echo-ed attentions for fr...