March 27, 2023

The GPT3 model from OpenAI is a new AI system that is surprising the world by its ability. This is a gentle and visual look at how it works under the hood -- including how the model is trained, and how it calculates its predictions.

Introduction & GPT-3 Demos (0:00)
GPT-3 Inputs and Outputs (2:06)
Training the GPT-3 model (2:48)
The scale of GPT-3 and its 175 billion parameters (6:37)
The order of GPT-3 token processing (7:58)
"Deep" learning: looking inside a layer stack (9:00)
Input prompts and priming examples (11:00)
Fine-tuning: the best is yet to come (11:56)