“Joint Embedding Predictive Architecture trained on images and videos produce good representations for image and video understanding:” Yann LeCun
CHENNAI : Dr. Yann LeCun, Vice-President and Chief AI Scientist at Meta, which owns Facebook, Instagram and WhatsApp, delivered Indian Institute of Technology Madras’ (IIT Madras) ‘Prof. Subra Suresh Institute Lecture Series’ in Chennai today (22nd Oct 2024).
Dr. Yann LeCun, also a faculty at New York University gave an address on the topic of ‘How Could Machines Reach Human-Level Intelligence?’.
Dr. Yann LeCun was awarded, along with Geoffrey Hinton and Yoshua Bengio, the 2018 Turing Award, often referred to as the ‘Nobel Prize in Computer Science’, for his work on Deep Learning.
The Lecture can be viewed using the following link – https://www.youtube.com/live/pjqKHOeykp8
IIT Madras launched the ‘Prof. Subra Suresh Institute Lecture Series’ during January 2020 in honour of one of its most illustrious alumni Prof. Subra Suresh, a Distinguished Alumnus Awardee (1997). It brings leading international scholars, engineers and scientists to IIT Madras on a periodic basis, up to twice every year, to discuss key advances in scientific discoveries and technological innovation.
Prof. Subra Suresh is a former Director of the U.S. National Science Foundation, former Dean of Engineering and Vannevar Bush Professor at Massachusetts Institute of Technology (MIT), and former President of Carnegie Mellon University. He is the first IIT alumnus and Indian-born academic appointed to these leadership positions.
This lecture series was made possible by a generous gift from another illustrious alumnus of IIT Madras, Mr. Senapathy ‘Kris’ Gopalakrishnan, Chairman of Axilor Ventures, Bengaluru, and co-founder of Infosys.
Delivering the lecture, Dr. Yann LeCun, Vice-President and Chief AI Scientist, Meta, said, “Animals and humans understand the physical world, have common sense, possess a persistent memory, can reason, and can plan complex sequences of sub-goals and actions. These essential characteristics of intelligent behavior are still beyond the capabilities of today’s most powerful AI architectures, such as Auto-Regressive LLMs (Large Language Models).”
Presenting a cognitive architecture during the lecture, Dr. Yann LeCun said, “This cognitive architecture may constitute a path towards human-level AI. The centerpiece of the architecture is a predictive world model that allows the system to predict the consequences of its actions. and to plan sequences of actions that fulfill a set of objectives. The objectives may include guardrails that guarantee the system’s controllability and safety. The world model employs a ‘Joint Embedding Predictive Architecture’ (JEPA) trained with self-supervised learning, largely by observation.”
Dr. Yann LeCun added, “The JEPA simultaneously learns an encoder, that extracts maximally-informative representations of the percepts, and a predictor that predicts the representation of the next percept from the representation of the current percept and an optional action variable. We show that JEPAs trained on images and videos produce good representations for image and video understanding. We show that they can detect unphysical events in videos. Finally, we show that planning can be performed by searching for action sequences that produce predicted endstate that match a given target state.”
The corresponding working paper can be accessed from the following link – https://openreview.net/forum?id=BZ5a1r-kVsf
Speaking earlier, Prof. V. Kamakoti, Director, IIT Madras, said, “With the advent of large language models, the edge defining the difference between artificial intelligence and natural intelligence is narrowing down. This interaction with Dr. Yann Lecun elaborated on important factors leading to this transformation.”
Commenting on the lecture series, Prof. Subra Suresh said, “Yann is a pioneer in the field of AI which is poised to significantly accelerate the way our lives and professions would be transformed in the years and decades to come. I am pleased that this lecture has provided a unique opportunity for the students of IIT Madras and the broader Chennai community to listen to, and interact with, one of the leading global thought-leaders in this field.”
Mr. Kris Gopalakrishnan, a Distinguished Alumnus Awardee, added, “I support this lecture series because IIT Madras, through such knowledge sessions, can contribute to society’s benefit. It should also spur new research collaborations in India.”
ABOUT PROF SUBRA SURESH: He joined IIT Madras at the age of 16 and received his B.Tech. degree in Mechanical Engineering in 1977 in first class with distinction. He was the Chief Guest of Honour and Convocation Keynote Speaker at the Golden Jubilee Convocation of IIT Madras in 2013, when he also received an honorary doctorate degree.
Nominated by the President of the United States and unanimously confirmed by the United States Senate, Prof Suresh was appointed as Director of the U.S. National Science Foundation (NSF) in 2010.
During his tenure as NSF Director, where he oversaw an annual budget of $7 billion in support of research in all fields of science and engineering, Prof. Suresh established a number of impactful initiatives including the NSF Innovation Corps (I-Corps) and Global Research Council.
Prof. Suresh was awarded the National Medal of Science, the highest honour accorded to a scientist by the President of the United States at a ceremony at the White House in October 2023.