Broader course for all
Scalable Artificial Intelligence
Markus Götz (KIT)
Abstract
Artificial intelligence methods have led to astonishing breakthroughs in science and technology over the past decade. In this context, there is an increasing trend towards processing ever larger amounts of data and the use of parallel and distributed computing resources. A prominent example is the machine translation algorithm Generative Pre-trained Transformer 3 (GPT-3) which pushes the limits of conventional AI hardware with 175 billion trainable parameters on 285,000 processor cores and 10,000 graphics cards. In the lecture, the audience is introduced to scalability approaches of different AI algorithms. The focus is on the advantages and approaches of parallel computing for AI methods, different available software packages for their implementation as well as algorithm-specific challenges. In particular, we will have deeper look on different flavors of parallel neural networks as well as scalable hyperparameter optimization.
