Getting Started with LLMs and Scary Moment

AI tools for software engineers, but without the hype – with Simon Willison (Co-Creator of Django)

Topic 2 of 7

Published Sep 25, 2024 · Clip 3:58–9:38 (5:40)

Simon recounts his ML journey: the fast.ai course in 2018, experimenting with GPT-2 to generate NYT headlines, and GPT-3 in 2020–2021 which was the first useful model. He used it for jq via completion-style prompting, noting ChatGPT's big innovation was the chat interface. OpenAI itself didn't expect code to be a strength. He then tells the story of ChatGPT's code interpreter mode flawlessly answering a SQLite question, which triggered existential dread since answering questions of databases was the whole point of his Datasette project, prompting him to add AI features to his own software.