US Patent Issued to Apple on April 8 for "Attention free transformer" (California Inventors)

ALEXANDRIA, Va., April 9 -- United States Patent no. 12,271,791, issued on April 8, was assigned to Apple Inc. (Cupertino, Calif.).

"Attention free transformer" was invented by Shuangfei Zhai (Sunnyvale, Calif.), Walter A. Talbott (Cupertino, Calif.), Nitish Srivastava (San Francisco), Chen Huang (Cupertino, Calif.), Hanlin Goh (Santa Clara, Calif.) and Joshua M. Susskind (Campbell, Calif.).

According to the abstract* released by the U.S. Patent & Trademark Office: "Attention-free transformers are disclosed. Various implementations of attention-free transformers include a gating and pooling operation that allows the attention-free transformers to provide comparable or better results to those of a standard attention-based transformer, with...