Towards Efficient and Effective AGI System

Artificial General Intelligence (AGI) aims to empower machines with human-like comprehension of language, speech, and vision data. While language and speech AI models like GPT and wave2vec have made remarkable strides, visual understanding, particularly before 2019, lags behind. This presentation aims to address the challenge of building an effective intelligent vision system to overcome AGI's visual limitations, with a specific focus on three core components: network architecture, learning framework, and parameter optimizer.

For network architectures, the speaker will introduce his MetaFormer and IFormer networks that possess high capacity and efficiency for acquiring vision knowledge, thereby improving the visual comprehension capacity and performance of the AI/AGI system.  Regarding the learning framework, he will present his self-supervised multi-granular learning framework that enables the AI model to learn general vision knowledge from large-scale data for addressing different vision tasks and thus bringing AGI closer. To enhance AGI’s efficiency, he will discuss his proposed Adan optimizer and Win acceleration. Adan is about 2x faster than SoTA optimizers, e.g. SGD and Adam on at least 15 kinds of networks, e.g. GPT2 and 7B-sized LLAMA model.