Lecturers
Each Lecturer will hold up to four lectures on one or more research topics.
Topics
Deep Learning, Multimodal Learning, Self-supervised Learning, Large Model Adaptation, LLMsBiography
Professor, University of Technology Nuremberg – formerly at UvA, VGG, FAIR
He is the head of the Fundamental AI (FunAI) Lab and full Professor at the University of Technology Nuremberg. Prior to this, I lead the QUVA lab at the University of Amsterdam, where I closely collaborated with Qualcomm AI Research. My PhD was at the Visual Geometry Group (VGG) at the University of Oxford, where I worked with Andrea Vedaldi and Christian Rupprecht. Also, I love running, the mountains and their combination.
Lectures
Abstract TBA
Abstract TBA
Abstract TBA
Topics
Artificial Intelligence, Deep Learning.Biography
Pierre Baldi is a chancellor’s professor of computer science at University of California Irvineand the director of its Institute for Genomics and Bioinformatics.
Pierre Baldi received his Bachelor of Science and Master of Science degrees at the University of Paris, in France. He then obtained his Ph.D. degree in mathematics at the California Institute of Technology in 1986 supervised by R. M. Wilson.
From 1986 to 1988, he was a postdoctoral fellow at the University of California, San Diego. From 1988 to 1995, he held faculty and member of the technical staff positions at the California Institute of Technology and at the Jet Propulsion Laboratory, where he was given the Lew Allen Award for Research Excellence in 1993. He was CEO of a start up company called Net-ID from 1995 to 1999 and joined University of California, Irvine in 1999.
Baldi’s research interests include artificial intelligence, statistical machine learning, and data mining, and their applications to problems in the life sciences in genomics, proteomics, systems biology, computational neuroscience, and, recently, deep learning.
Baldi has over 250 publications in his field of research and five books including
- “Bioinformatics: the Machine Learning Approach” (MIT Press, 1998; 2nd Edition, 2001, ISBN 978-0262025065) a worldwide best-seller
- “Modeling the Internet and the Web. Probabilistic Methods and Algorithms“, by Pierre Baldi, Paolo Frasconi and Padhraic Smyth. Wiley editors, 2003.
- “The Shattered Self—The End of Natural Evolution“, by Pierre Baldi. MIT Press, 2001.
- “DNA Microarrays and Gene Regulation“, Pierre Baldi and G. Wesley Hatfield. Cambridge University Press, 2002.
- “Deep Learning in Science”, Pierre Baldi, Cambridge University press, 2021.
Baldi is a fellow of the Association for the Advancement of Artificial Intelligence (AAAI), the AAAS, the IEEE,and the Association for Computing Machinery (ACM). He is also the recipient of the 2010 Eduardo R. Caianiello Prize for Scientific Contributions to the field of Neural Networks and a fellow of the International Society for Computational Biology (ISCB).
Deep learning algorithm solves Rubik’s Cube faster than any human.
AI solves Rubik’s Cube in one second
https://www.bbc.com/news/technology-49003996
https://en.wikipedia.org/wiki/Pierre_Baldi
https://scholar.google.com/citations?user=RhFhIIgAAAAJ&hl=it
Lectures
Abstract TBA
Abstract TBA
Topics
Foundation Models, Transformers, Representation Learning, Reinforcement Learning,Biography
Lucas grew up in Belgium wanting to make video games and their AI. He went on to study mechanical engineering at RWTH Aachen in Germany, then did a PhD in robotic perception and computer vision there too. He was a staff research scientist at Google DeepMind (formerly Brain) in Zürich, leading multimodal vision-language research. He establish OpenAI’s Zürich office and research team.
https://www.s-ge.com/en/article/news/20244-ai-openai-subsidiary-zurich?ct
https://www.wired.com/story/openai-hires-deepmind-zurich/
https://scholar.google.com/citations?user=p2gwhK4AAAAJ
Lectures
Abstract TBA
Abstract TBA
Abstract TBA
Abstract TBA
Topics
Foundation Models, AI for ScienceBiography
My name is Wessel, and I am a Senior Researcher on the AI for Science initiative at Microsoft Research Amsterdam(opens in new tab), researching foundation models for environmental forecasting. My team recently published Aurora(opens in new tab), a foundation model of the atmosphere, which was featured in a Nature new article(opens in new tab).
I hold a PhD in Machine Learning from the Machine Learning Group at the University of Cambridge(opens in new tab), supervised by Richard Turner(opens in new tab). During the PhD, I worked mainly on neural processes and Gaussian processes.
I also like to work on various open-source software projects(opens in new tab).
Lectures
Topics
White-Box Deep Networks, White-box transformers, Machine LearningBiography
Sam Buchanan (TTIC, sam@ttic.edu) is a Research Assistant Professor at the Toyota Technological Institute at Chicago (TTIC). He obtained his Ph.D. in Electrical Engineering from Columbia University in 2022. His research develops the mathematical foundations of representation learning for high-dimensional data, and applies these principles to design scalable, transparent, and efficient deep architectures for problems in imaging and computer vision. He received the 2017 NDSEG Fellowship, and the 2022 Eli Jury Award from Columbia University.
Lectures
Topics
Graph Learning, Graph Neural NetworksBiography
Floris Geerts received his M.Sc. in Mathematics from the University of Ghent (Belgium) in 1995, and a Ph.D. in Computer Science from the University of Hasselt (Belgium) in 2001. He has been postdoctoral researcher at the University of Helsinki (Finland) and was Senior Research Fellow at the University of Edinburgh (UK) for seven years. He joined the University of Antwerp as Research Professor in 2011. His research interests include graph learning and graph neural networks.
Lectures
Abstract TBA
Abstract TBA
Topics
Foundation Models, Large Language Models, Natural Language Understanding, Deep Learning,Biography
Sven Giesselbach is the leader of the Natural Language Understanding (NLU) team at the Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS). His team develops solutions in the areas of medical, legal and general document understanding which in their core build upon (large) pre-trained language models. Sven Giesselbach is also part of the Lamarr Institute and the OpenGPT-X project in which he investigates various aspects of Foundation Models. Based on his project experience of more than 25 natural language understanding projects he studies the effect of Foundation Models on the execution of Natural Language Understanding projects and the novel challenges and requirements which arise with them. He has published several papers on Natural Language Processing and Understanding, which focus on the creation of application-ready NLU systems and the integration of expert knowledge in various stages of the solution design. Most recently he co-authored a book on “Foundation Models for Natural Language Processing – Pre-trained Language Models Integrating Media” which will be published by Springer Nature.
Gerhard Paaß, Sven Giesselbach, Foundation Models for Natural Language Processing – Pre-trained Language Models Integrating Media, Springer, May, 2023
Lectures
Abstract TBA
Abstract TBA
Topics
Diffusion Models, Machine Learning, Deep Learning, Probabilistic Inference, Generative ModelingBiography
I am a senior researcher at MSR Amsterdam(opens in new tab), working on machine learning for molecular simulation. I did my PhD in computer science at Mila(opens in new tab) (University of Montreal) with Aaron Courville(opens in new tab). I have worked on a wide variety of topics in core ML, including generative models, variational inference, and Bayesian deep learning. I received a Google PhD fellowship(opens in new tab) in the category of Machine Learning in 2020. Throughout my PhD, I also spent some time interning at Google and Element AI (acquired by ServiceNow), and helped organize INNF+(opens in new tab), a workshop on invertible flows and other likelihood-based models from 2019 to 2021. Prior to my PhD, I obtained my Bachelor’s degree in chemical engineering at National Taiwan University(opens in new tab) (NTU).
Lectures
Abstract TBA
Topics
Computer Vision, Multimodality, Diffusion Models, Generative AI.Biography
Vicky Kalogeiton (HDR) has been an Assistant Professor at École Polytechnique since 2020. Before, she was a research fellow at the University of Oxford, working with A.Zisserman. In 2017, she obtained her PhD from the University of Edinburgh and Inria, Grenoble, advised by V.Ferrari and C.Schmid, where part of her thesis won the best poster award from the Grenoble Alpes University. She received her M.Sc degree in Computer Science from DUTh, Greece in 2013, being awarded the best master thesis award. Since 2021, V.Kalogeiton has received several awards for projects she supervised including a highlight at CVPR 2024, a student honorable mention award at ACCV 2022, and the best paper award at ICCV-W 2021 and grants, including two MS Azure Academic gifts, and an ANR JCJC award for junior researchers in France. She was Associate Editor for CMBBE from 2017 to 2024 and has been Associate Editor for CVIU since 2024. Since 2021, she has been serving regularly as Area Chair at all major vision conferences (outstanding Area Chair in 2022) and before she used to serve as a reviewer, having been awarded six times as an outstanding reviewer. Her research focuses on generative AI using multiple modalities (visual data, text, audio, trajectories).
Lectures
Topics
Graph Neural Networks, Machine Learning, Deep LearningBiography
Thomas Kipf is a Research Scientist at Google Research in the Brain Team in Amsterdam. Prior to joining Google, he completed his PhD at University of Amsterdam under Prof. Max Welling on the topic “Deep Learning with Graph-Structured Representations”. His research interests lie in the area of relational learning and in developing models that can reason about the world in terms of structured abstractions such as objects or events.
Lectures
Abstract TBA
Abstract TBA
Topics
AI, Computer Vision, Compressed Sensing, Machine Learning, Signal Processing, RoboticsBiography
Education: Yi Ma received his Bachelor’s degree in Automation and a second degree in Applied Mathematics from Tsinghua University, Beijing, China, in 1995. He received an Master degree in Electrical Engineering and Computer Sciences (EECS) in 1997, a second Master degree in Mathematics in 2000, and the Ph.D. degree in EECS in 2000, all from the University of California at Berkeley.
Academia: From 2000 to 2011, he served as an assistant and associate professor of the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, where he now holds an Adjunct Professorship. He also served as a research professor both in the Decision & Control Group of the Coordinated Science Laboratory and in the Image Formation & Processing Group of the Beckman Institute. He was a visiting professor at EECS Department of UC Berkeley during sabbatical in spring 2007. From February 2014 to November 2017, he was a professor and then the executive dean of the School of Information Science and Technology of ShanghaiTech University. While he was working in China, he had also held several courtesy appointments as Adjunct Professor at the University of Electronic Science and Technology, the Univeristy of Science and Technology, and Shanghai Jiao Tong University, while working in China. He has joined the faculty of the EECS Department of University of California at Berkeley since January 2018. Starting in Januaray 2023, he serves as the inaugural director of the Institute of Data Science of the University of Hong Kong. He also served as the head of the Computer Science Department of the University of Hong Kong from June 2023 to July 2024. Starting in July 1st 2024, he serves as the inaugural director of the School of Computing and Data Science of the Hong Kong University.
Industrial: He was a visiting senior researcher at the Microsoft Research Asia, Beijing, China, during sabbatical in fall 2006. From January 2009 to January 2014, he served as the research manager and principal researcher of the Visual Computing Group at Microsoft Research Asia, Beijing, China. He is a co-founder of a light-field 3D acquisition startup company DGene Digital Technology (previously known as Plex-VR), established in early 2016. He has served as senior advisor to the Bytedance Research Lab in Silicon Valley from 2017 to 2020. He is on the Technical Advisorial Board (TAB) of Malong Technologies (a computer vision startup company in Shenzhen, China), since June 2018. He also serves as an independent director on the Board of Directors of Cheetah Mobile Inc. (NYSE: CMCM), since March 2018.
Academic Services: He has served as an associate editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) from 2007 to 2011, the International Journal of Computer Vision (IJCV) from 2010 to 2014, the IEEE Transactions on Information Theory from 2013 to 2017, and a founding associate editor of SIAM Journal on Imaging Sciences. He served on the Senior Editorial Board of IEEE Signal Processing Magazine from 2015 to 2017. He has also served as the chief guest editor for special issues for the Proceedings of IEEE and the IEEE Signal Processing Magazine in 2010 and 2011, respectively. He currently serves as a founding associate editor for the IMA Journal on Information and Inference since 2012 and for the SIAM Journal on Mathematics of Data Science (SIMODS) since 2018. For conferences, he has served many times as Area Chairs for ICCV, CVPR, and NIPS, and the Program Chair for ICCV 2013 (Australia), and General Chair for ICCV 2015 (Chile). He is one of the co-founders of the new Conference on Parsimony and Learning (CPAL), starting in 2024.
https://people.eecs.berkeley.edu/~yima/Biography.html
Lectures
Topics
LLMs, Foundation Models, AI, NLP.Biography
Raniero Romagnoli is CTO of Almawave, VP of PerVoice and CEO of OBDA Systems. He is an expert in Artificial Intelligence and Natural Language Processing both in the corporate and academic world. He leads the company’s technological strategy by managing research and development and innovation teams. He actively participates in numerous national and international initiatives in the field of AI by collaborating with research centers and academies, he holds advanced courses in Data Science, Machine Learning and AI. He is also co-author of numerous scientific articles and international patents.
Lectures
Abstract TBA
Topics
Natural Language ProcessingBiography
Since January 2022, I am a Research Scientist at Google DeepMind in London. You can follow my post-Stanford work on my Scholar or X-Twitter page.
In September 2021 I graduated with a PhD in Computer Science from Stanford University, advised by Professor Chris Manning in the Natural Language Processing group.
My research focuses on understanding and improving neural text generation techniques for open-ended tasks, such as story generation and chitchat dialogue. In particular, I’m interested in improving the controllability, interpretablility and coherence of neural text generation, and applying it to real-world scenarios, such as our award-winning Alexa Prize chatbot.
I’m passionate about scientific communication. Whether through teaching, writing, or moderating debates, I aim to communicate technical concepts as accessibly as possible.
Sumanth Dathathri, Abigail See, …, Demis Hassabis & Pushmeet Kohli, “Scalable watermarking for identifying large language model outputs”, Nature volume 634, pages 818–823 (2024)
https://www.nature.com/articles/s41586-024-08025-4
Lectures
1. Detecting AI-generated text
2. SynthID: a watermark for LLM-generated text
3. Neural text generation: history to present
1. Detecting AI-generated text
2. SynthID: a watermark for LLM-generated text
3. Neural text generation: history to present
Sumanth Dathathri, Abigail See, …, Demis Hassabis & Pushmeet Kohli, “Scalable watermarking for identifying large language model outputs“, Nature volume 634, pages 818–823 (2024)
1. Detecting AI-generated text
2. SynthID: a watermark for LLM-generated text
3. Neural text generation: history to present
Topics
Foundation Models, Fine-Tuning Large Language Models, Reinforcement Learning with Human Feedback, Deep Reinforcement LearningBiography
Michal is the Chief Models Officer in a stealth startup, tenured researcher at Inria, and the lecturer at the MVA master of ENS Paris-Saclay. Michal is primarily interested in designing algorithms that would require as little human supervision as possible. That is why he is working on methods and settings that are able to deal with minimal feedback, such as deep reinforcement learning, bandit algorithms, self-supervised learning, or self play. Michal has recently worked on representation learning, word models and deep (reinforcement) learning algorithms that have some theoretical underpinning. In the past he has also worked on sequential algorithms with structured decisions where exploiting the structure leads to provably faster learning. Michal is now working on large large models (LMMs), in particular providing algorithmic solutions for their scalable fine-tuning and alignment. He received his Ph.D. in 2011 from the University of Pittsburgh under the supervision of Miloš Hauskrecht and was a postdoc of Rémi Munos before getting a tenure at Inria in 2012 and co-creating Google DeepMind Paris in 2018.
Lectures
Metacognitive knowledge refers to humans’ intuitive knowledge of their own thinking and reasoning processes. Today’s best LLMs clearly possess some reasoning processes. The paper gives evidence that they also have metacognitive knowledge, including ability to name skills and procedures to apply given a task. We explore this primarily in the context of math reasoning, developing a prompt-guided interaction procedure to get a powerful LLM to assign sensible skill labels to math questions, followed by having it perform semantic clustering to obtain coarser families of skill labels. These coarse skill labels look interpretable to humans.
To validate that these skill labels are meaningful and relevant to the LLM’s reasoning processes we perform the following experiments. (a) We ask GPT-4 to assign skill labels to training questions in math datasets GSM8K and MATH. (b) When using an LLM to solve the test questions, we present it with the full list of skill labels and ask it to identify the skill needed. Then it is presented with randomly selected exemplar solved questions associated with that skill label. This improves accuracy on GSM8k and MATH for several strong LLMs, including code-assisted models. The methodology presented is domain-agnostic, even though this article applies it to math problems.
Abstract (see Lecture 1/3).
Abstract (see Lecture 1/3).
Topics
Interpretable white-box deep neural networks, differentially private foundation models, white-box transformers, machine learning.Biography
Yaodong Yu (OpenAI, yyu@openai.com) is a Member of Technical Staff at OpenAI. He obtained his PhD from the EECS department at UC Berkeley advised by Michael I. Jordan and Yi Ma. He obtained his B.S. from the Department of Mathematics at Nanjing University, and his M.S. from the Department of Computer Science, University of Virginia. His main research interest is in building reliable machine learning systems, including transparency, robustness, and privacy.
Lectures
Tutorial Speakers
Each Tutorial Speaker will hold more than four lessons on one or more research topics.
Topics
Deep Learning Theory, NLP, Computer Vision, White-Box Transformers, Demystifying Extreme-Token Phenomena in LLMsBiography
Druv Pai (UC Berkeley, druvpai@berkeley.edu) is a PhD student in the EECS department at UC Berkeley advised by Yi Ma and Jiantao Jiao. He obtained his B.A. and M.S. from the EECS department at UC Berkeley. He received the 2023 UC Berkeley College of Engineering fellowship. His main research interests lie in understanding the statistics and geometry of deep neural networks, especially in the context of representation learning and generative models.
Lectures
Topics
mathematical foundations of deep learning models (supervised learning models, diffusion models and large language models).Biography
Peng Wang (University of Michigan, pengwa@umich.edu) is a postdoc research fellow at the University of Michigan, Ann Arbor. He received his PhD in System Engineering and Engineering Management from the Chinese University of Hong Kong. His main research interest lies in the intersection of optimization, machine learning, and data science. Currently, his work is devoted to understanding the mathematical foundations of deep learning models, including supervised learning models, diffusion models, and large language models. He received the CPAL 2024 Rising Star Award.