Computer Science Talk - David Patterson
OCT 17, 2019 | 2:00 PM TO 3:00 PM
The Graduate Center
365 Fifth Avenue
October 17, 2019: 2:00 PM-3:00 PM
Computer Science Department
David Patterson - Turing Award Winner, Google Distinguished Engineer
Domain Specific Architectures for Deep Neural Networks: Three Generations of Tensor Processing Units (TPUs)
The recent success of deep neural networks (DNN) has inspired a resurgence in domain specific architectures (DSAs) to run them, partially as a result of declaration of microprocessor performance improvement due to the ending in Moore’s Law.
DNN have two phases: training, which constructs accurate models, and inference, which serves those models. Google’s firs generations Tensor Processing Unit (TPUv1) offered 50x improvement in performance per watt over conventional architecture for inference. We naturally asked whether a successor could do the same training.
This talk reviews TPUv1 and explores how Google built the first production DSA supercomputer for the much harder problem of training, which was deployed in 2017.
Google’s TPUv2/TPUv3 supercomputers with up to 1024 chips train production DNNs at close to perfect linear speedup with 10X-40X higher floating point operations per Watt than general-purpose supercomputers running the high-performance computing benchmark Linpack.
BIO: David Patterson is a Berkeley CS Professor emeritus, a Google distinguished engineer, and the RISC-V Foundation Vice-Chair. He received his BA, MS, and Ph.D., degrees from UCLA. His Reduced Instruction Set Computer (RISC), Redundant Array of Inexpensive Disks (RAID), and Network of Workstation projects helped lead to multibillion-dollar industries. This work led to 40 awards for research, teaching, and service plus many papers and seven books. The best known books is Computer Architecture: A Quantitative Approach and the newest is The RISC-V Reader: An Open Architecture Atlas. In 2018 he and John Hennessy shared the ACM A.M. Turing Award.
365 Fifth Ave, New York City 10016 | Room 4319 | Phone: 212.817.8190 | Fax: 212.817.1510 | email@example.com