New Lex Fridman Insight: Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs
Sent June 11, 2026
Key Insights
- LSTMs are integral to billions of devices for tasks like speech recognition and translation.
- Meta-learning involves machines improving their own learning algorithms recursively, a concept Schmidhuber explored in 1987.
- The universe's randomness at the quantum level lacks physical evidence, challenging the notion of fundamental randomness.
- Reinforcement learning is crucial for AI applications like self-driving cars, enabling learning from interactions without supervision.
- Countries with high robot density like Japan have low unemployment, suggesting automation leads to new job creation.
How the conversation moved
Lex Fridman opened the conversation by framing the discussion around the evolution of AI technologies, specifically focusing on LSTMs and meta-learning. Jürgen Schmidhuber, a pioneer in the field, introduced the concept of meta-learning, which he had explored as early as 1987. He explained that meta-learning involves a machine's ability to improve its own learning algorithms recursively, a process that could potentially lead to general AI. Schmidhuber also highlighted the widespread use of LSTMs in billions of devices today, underscoring their significance in tasks like speech recognition and translation.
Schmidhuber argued that the deterministic nature of the universe, as seen in the pseudo-randomness of pi, challenges the notion of fundamental randomness at the quantum level. He suggested that scientific theories have historically evolved towards greater simplicity and compression, reflecting a trend towards more elegant and predictive models. This perspective aligns with his view that the universe could be described by a short program, making it more beautiful and comprehensible. Schmidhuber also discussed the importance of reinforcement learning in AI applications, particularly in autonomous systems like self-driving cars.
Despite the compelling arguments, Lex did not challenge Schmidhuber's views on quantum randomness, which could have been a point of contention given the prevailing scientific consensus on quantum mechanics. The conversation lacked explicit pushback, particularly on the feasibility of meta-learning leading to general AI, a topic that remains highly debated within the AI community. The absence of pushback left some of Schmidhuber's claims unexamined, such as the practicality of implementing meta-learning in current AI systems.
The discussion concluded with Schmidhuber's optimistic view of AI's future, particularly its potential societal and economic impacts. He noted that countries with high robot density, like Japan, have low unemployment rates, suggesting that automation may lead to job creation rather than loss. Schmidhuber also speculated on the possibility of advanced civilizations in the universe, pondering humanity's role in this broader context. The conversation ended with an open question about the future of AI and its implications for the universe, leaving listeners with much to consider about the trajectory of technology and its integration into society.
Surprising moments
In-depth
LSTMs and Meta-Learning
- LSTMs are used in billions of devices for tasks like speech recognition.
- Schmidhuber's 1987 thesis on meta-learning involves recursive improvement of learning algorithms.
- Transfer learning differs from meta-learning by focusing on adapting pre-trained models to new data.
Determinism and Randomness
- Zeilinger's claim of quantum randomness lacks physical evidence.
- Pi's pseudo-randomness suggests deterministic patterns in seemingly random sequences.
- Scientific theories evolve towards simplicity and compression over time.
Reinforcement Learning and Creativity
- Applied creativity involves solving human-defined problems, while pure creativity involves self-defined problems.
- RNNs create predictive models aiding in data compression.
- Reinforcement learning is crucial for self-driving cars and robotics.
AI's Societal Impact
- AI currently represents a small fraction of the global economy but could grow significantly.
- High robot density correlates with low unemployment, suggesting automation creates jobs.
- The universe's age suggests ample time for AI expansion throughout it.
Notable Quotes
Experience tells us that the stuff that works best is really simple.
Still open
- Schmidhuber speculated on whether humanity is the first advanced civilization, leaving open the question of our significance in the universe.
- The feasibility of implementing meta-learning in current AI systems remains an open question, as discussed by Schmidhuber.