Most people know these results second-hand. Each is short enough to read in an evening, and each still explains a design decision in today’s systems.
- Attention Is All You Need (2017). The architecture under almost everything.
- Scaling Laws (2020). Why labs were willing to spend so much on compute.
- Retrieval-Augmented Generation (2020). The pattern behind most company chatbots.
- InstructGPT (2022). How a raw model becomes an assistant.
- Chain-of-Thought (2022). The seed of today’s reasoning models.
- Chinchilla (2022). Why data matters as much as parameters.
- ReAct (2022). The agent loop, in its original form.
- Constitutional AI (2022). Training behaviour from written principles.
Read them in order and the last eight years of AI make a lot more sense.