| |
 Scalable and Efficient AI: From Supercomputers to Smartphones Speaker (s):
Torsten Hoefler
Chief Architect,
Machine Learning at Swiss National
Supercomputing Centre
| Date: Time: Venue: | | 13 January 2025, Monday 10:00am – 11:00am School of Computing &
Information Systems 2 (SCIS 2)
Level 3, Seminar Room 3-8
Singapore Management University
90 Stamford Road,
Singapore 178903 Please register by 10 January 2025. We look forward to seeing you at this research seminar. 
|
|
About the Talk We will explore the Age of Computation, delving into the potential of machines to surpass Human Intelligence and Creativity. The discussion will trace the evolution of large language models from the inception of transformers to contemporary advanced reinforcement learning methodologies. We will analyze the recent plateau in model scaling and investigate the pivotal role of high-performance computation in both AI training and inference. A significant portion of the talk will focus on the escalating cost of data movement in modern computing systems and strategies to leverage this for code optimization. We will examine techniques to compress (quantize and sparsify) weights, leading to over 100x reductions in model size and improved computational efficiency.
This will be complemented by networking optimizations that offer an additional 10x cost savings. Moreover, we will explore innovative inference-based approaches fostering the development of modern reasoning models. These models, built upon reinforcement learning and employing chains or graphs of thoughts, signify a progression towards AGI-like reasoning agents. The session will conclude with a deep dive into industry trends, specifically the development of Ultra Ethernet—an advanced interconnect standard designed to support AI and HPC computations, paving the way for larger and more cost-effective systems.
This comprehensive examination will provide insights into the future of AI and computation, highlighting the advancements and challenges that lie ahead. About the Speaker Prof. Hoefler is a full professor at ETH Zurich and directs the Scalable Parallel Computing Laboratory. He serves as Chief Architect for Machine Learning at the Swiss National Supercomputing Centre and as a consultant for Microsoft, focusing on AI and networking.
Renowned for his contributions to high-performance computing, he examines parallel computing systems and enhances scientific simulations in areas like weather prediction and distributed deep learning.
His accolades include the IEEE CS Sidney Fernbach Memorial Award, the ACM Gordon Bell Prize, and the 2024 Max Planck-Humboldt Model, one of Germany’s highest honours for foreign scientists. |
