Since their introduction in 2017, Transformers have become the de facto standard for tackling a wide range of NLP tasks in both academia and industry. In this workshop, Lewis Tunstall and Leandro von Werra teach you the core concepts behind Transformers and how to train these models in the Hugging Face ecosystem.
00:02:23 Plan of attack
00:04:33 Transformers in the wild
00:06:38 What is a Transformer? 🤖
00:20:32 Three types of architectures
00:24:48 The modern paradigm
00:31:55 Main challenges
00:36:11 Can we do better?
00:37:32 The 🤗 Ecosystem
00:44:49 Notebook 1: Transformers tour
01:24:49 Notebook 2: Fine-tuning your first Transformer
02:22:07 The Hub
02:25:26 Notebook 3: Spaces – Building ML applications with a few lines of code
Lewis Tunstall is a machine learning engineer at Hugging Face, focused on developing open-source tools and making them accessible to the wider community. A former theoretical physicist, he has over 10 years experience translating complex subject matter to lay audiences and has taught machine learning to university students at both the graduate and undergraduate levels.
Leandro von Werra is a machine learning engineer at Hugging Face. He has several years of industry experience bringing NLP projects to production by working across the whole machine learning stack, and is the creator of a popular Python library called TRL that combines Transformers with reinforcement learning.