Dr. Imanol Schlag - Pioneering the Future of AI - Language Models and the Swiss AI Initiative

In this episode, host Lauren Hawker Zafer welcomes Dr. Imanol Schlag, a notable AI and machine learning researcher currently at ETH Zurich. The discussion begins with an overview of Dr. Schlag's academic background, including his PhD focused on advanced neural network architectures under Jürgen Schmidhuber at the Swiss AI Lab. He has achieved recognition for his innovative research on language models and quantitative reasoning, particularly in linear transformers and fast weight programming.

Dr. Schlag shares his personal journey into artificial intelligence, highlighting an early fascination with solving problems, which eventually led him to study computer science. His experiences during an apprenticeship and military service shaped his understanding of the field. A pivotal moment was his mastery project on identifying Roman emperors through neural networks, igniting his passion for deeper AI research and leading to his PhD.

Reflecting on the evolving landscape of AI from 2016 onwards, Dr. Schlag discusses the dramatic shifts, particularly the move towards generative AI. He emphasizes how advances in computational power and data availability have fueled growth in deep learning, moving from predictive models to generative models, such as ChatGPT. He highlights the importance of efficient algorithms like linear transformers in shaping future models.

Dr. Schlag elaborates on his current work with the Swiss National AI Initiative, co-leading efforts on developing large language models (LLMs) that are not only effective but also transparent and trustworthy. The initiative aims to harness a vast dataset to reflect Swiss culture and needs while remaining compliant with European regulations on AI. He discusses the significance of creating publicly accessible models to democratize AI technology.

A significant theme throughout the conversation is the need for transparency and trustworthiness in AI development. Dr. Schlag expresses concerns about variations in regulatory approaches across countries, emphasizing the necessity for compliance to foster innovation without compromising ethics. He articulates a vision where AI remains accessible and adaptable for diverse applications across various sectors.

The episode concludes with discussions on the potential for public models to foster innovation and adaptability within organizations. Dr. Schlag envisions a future where advanced AI technologies empower small and medium enterprises and governmental institutions, allowing them to tailor tools according to their needs, thus promoting the democratization of AI.