If you’ve ever read about modern physics, you know your high school physics math won’t get you very far. Books by Stephen Hawking and Brian Greene contain descriptions that you have to take on faith. Without the math, all that science may as well be science fiction. Computing has reached a similar place with artificial intelligence. Unless you know a lot about bits of architecture in software called “transformers,” you have to listen to sales pitches by tech CEOs and try to separate the big deals from bullshit.
Transformers were popularized in a now-iconic 2017 paper by eight Google scientists called “Attention Is All You Need.” It’s important to know a few things about them: They parse a whole sentence at once; they convert words into interchangeable chunks of code in a way that makes AI good at analogy, but at least so far, bad at counting the number of r’s in ‘strawberry;’and, importantly, they don’t seem to …