Hauptinhalt

Topinformationen

Personen

Building GPT from scratch

8.30803

Dozenten

Beschreibung

This intensive block course offers a unique opportunity to build a Generatively Pretrained Transformer (GPT) from the ground up, closely following the foundational paper "Attention is All You Need" and OpenAI's GPT-2 and GPT-3 models. Students will gain a comprehensive understanding of how state-of-the-art language models work, from the multi-head attention mechanism to positional encoding and transformer-based architectures. Connections to modern applications such as ChatGPT will also be explored.

The course is structured as a hackathon, demanding significant dedication and active participation. Over the course of this hands-on experience, students will implement core components of GPT using PyTorch, delve into optimization and training strategies, and tackle practical challenges associated with scaling.

This block course is ideal for highly motivated students who are prepared to work hard and immerse themselves in the challenges of implementing a cutting-edge AI model. By the end of the hackathon, participants will have gained both theoretical insights and practical experience with one of the most transformative technologies in modern AI.

Weitere Angaben

Ort: 11/116
Zeiten: Termine am Montag, 21.07.2025 12:00 - 18:00, Dienstag, 22.07.2025 09:00 - 12:00, Dienstag, 22.07.2025 13:00 - 16:00, Mittwoch, 23.07.2025 09:00 - 12:00, Mittwoch, 23.07.2025 13:00 - 16:00, Donnerstag, 24.07.2025 09:00 - 12:00, Donnerstag, 24.07.2025 13:00 - 16:00, Freitag, 25.07.2025 09:00 - 12:00, Freitag, 25.07.2025 13:00 - 16:00, Montag, 28.07.2025 12:00 - 18:00, Dienstag, 29.07.2025 09:00 - 12:00, Dienstag, 29.07.2025 13:00 - 16:00, Mittwoch, 30.07.2025 09:00 - 12:00, Mittwoch, 30.07.2025 13:00 - 16:00, Donnerstag, 31.07.2025 09:00 - 12:00, Donnerstag, 31.07.2025 13:00 - 16:00, Freitag, 01.08.2025 09:00 - 12:00, Freitag, 01.08.2025 13:00 - 16:00
Erster Termin: Montag, 21.07.2025 12:00 - 18:00, Ort: 11/116
Veranstaltungsart: Blockseminar (Offizielle Lehrveranstaltungen)