https://www.quantamagazine.org/chatbot-software-begins-to-face-fundamental-limitations-20250131/

A fundamental limit of our current architecture would be good news.

“in December 2024, Peng and colleagues at the University of California, Berkeley posted a proof … showing that multilayer transformers indeed cannot solve certain complicated compositional tasks.”