the.com/transformer
the machine that learned to read everything by paying attention to nothing in particular
means A transformer is either an electrical device that steps voltage up or down by passing alternating current between coupled coils, or the neural-network architecture that processes data by weighing how every piece relates to every other piece (the 'attention' mechanism behind modern AI).
from From the verb 'transform,' which came through Old French 'transformer' from Latin 'transformare' — 'trans-' (across, over) plus 'formare' (to shape), literally to reshape across into a new form. The electrical sense arrived in the late 19th century for the device that transforms voltage. The AI sense is much newer, named in a 2017 research paper titled 'Attention Is All You Need,' chosen because the model transforms one sequence into another.
- the paper
- Named after one line: Attention Is All You Need
- powers chatgpt
- The T in GPT means transformer
- no recurrence
- Reads whole sequences at once, not word by word
- born 2017
- Google researchers reshaped AI in eight pages
- attention math
- Every word secretly weighs every other word