🤗 Transformers

Since their introduction in 2017, Transformers have become the de facto standard for tackling a wide range of NLP tasks in both academia and industry. In this workshop, Lewis Tunstall and Leandro von Werra teach you the core concepts behind Transformers and how to train these models in the Hugging Face ecosystem.

​

00:00:00 Introduction

00:02:23 Plan of attack

00:04:33 Transformers in the wild

00:06:38 What is a Transformer? 🤖

00:20:32 Three types of architectures

00:24:48 The modern paradigm

00:31:55 Main challenges

00:36:11 Can we do better?

00:37:32 The 🤗 Ecosystem

00:44:49 Notebook 1: Transformers tour

01:24:49 Notebook 2: Fine-tuning your first Transformer

02:22:07 The Hub

02:25:26 Notebook 3: Spaces – Building ML applications with a few lines of code

​

Workshop Instructors

​

Lewis Tunstall is a machine learning engineer at Hugging Face, focused on developing open-source tools and making them accessible to the wider community. A former theoretical physicist, he has over 10 years experience translating complex subject matter to lay audiences and has taught machine learning to university students at both the graduate and undergraduate levels.

​

Leandro von Werra is a machine learning engineer at Hugging Face. He has several years of industry experience bringing NLP projects to production by working across the whole machine learning stack, and is the creator of a popular Python library called TRL that combines Transformers with reinforcement learning.