Transformer解释器：文本生成模型的交互式学习

Transformer Explainer: Interactive Learning of Text-Generative Models

August 8, 2024

作者: Aeree Cho, Grace C. Kim, Alexander Karpekov, Alec Helbling, Zijie J. Wang, Seongmin Lee, Benjamin Hoover, Duen Horng Chau

cs.AI

摘要

Transformer已经彻底改变了机器学习，然而对许多人来说，它的内部运作仍然是不透明的。我们推出了Transformer Explainer，这是一个交互式可视化工具，专为非专家设计，通过GPT-2模型来学习Transformer。我们的工具帮助用户理解复杂的Transformer概念，通过整合模型概述，并实现在数学操作和模型结构的抽象层级之间平滑过渡。它在用户的浏览器中本地运行一个实时的GPT-2实例，使用户能够尝试他们自己的输入，并实时观察Transformer的内部组件和参数如何共同预测下一个标记。我们的工具无需安装或特殊硬件，扩大了公众对现代生成式人工智能技术的教育获取。我们的开源工具可在https://poloclub.github.io/transformer-explainer/找到。视频演示请访问https://youtu.be/ECR4oAwocjs。

English

Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present Transformer Explainer, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and enabling smooth transitions across abstraction levels of mathematical operations and model structures. It runs a live GPT-2 instance locally in the user's browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. Our tool requires no installation or special hardware, broadening the public's education access to modern generative AI techniques. Our open-sourced tool is available at https://poloclub.github.io/transformer-explainer/. A video demo is available at https://youtu.be/ECR4oAwocjs.