Transformer

This is the neural network architecture Google researchers introduced in 2017 in the paper 'Attention Is All You Need.' Unlike earlier methods that read a sentence one word at a time in order, it looks at the whole sentence at once and calculates how much each word relates to every other word — this is called attention.

Because this structure is so well suited to parallel computation, it could be scaled up massively on GPUs, and the result is today's language models. The T in GPT stands for Transformer — ChatGPT, Claude, and Gemini are all built on this same backbone. Articles about 'a new architecture that will replace the Transformer' pop up regularly, but for now, the throne hasn't changed hands.