005: Fable 5 is here! · Token Town summary

Key takeaways

Fable 5 is a significant leap forward in AI code generation, particularly excelling in building complex modules and handling large projects.
Model restrictions, especially around security and biological threats, reflect ongoing safety concerns but also limit certain use cases for developers.
The industry trends show increasing use of non-US, especially Chinese, models through platforms like OpenRouter, though their quality for coding tasks remains secondary.
New features like nested sub-agents and loops suggest a move towards enterprise-scale automation, but their practical benefit for small developers is questionable.
Benchmarking in AI remains contentious, with questions about how to measure real-world utility versus theoretical performance.
Market strategies seem geared toward large institutions with unlimited budgets, raising concerns about accessibility and safety.
The overall tone indicates a mix of excitement and skepticism about the rapid proliferation of advanced AI capabilities and their societal implications.

Summary

Introduction and Sponsorship Acknowledgment

The hosts, Ian Lansman and Aaron Francis, introduce the episode and thank their sponsor, Bento, an email marketing and CRM platform optimized for AI applications. They highlight Bento's free initial offering and their personal positive experiences with the platform.

Deep Dive into Fable 5 and Mythos 5

The hosts discuss the release of Fable 5 and Mythos 5, emphasizing their advanced capabilities and improvements over previous models. They examine how Fable 5 performs, especially its code generation abilities, and compare it to older models like Opus 4.8. Ian is impressed with its performance on large projects, and Aaron notes limitations and restrictions, particularly around security and biological weapons.

Model Capabilities and Limitations

The conversation explores model restrictions, plan limits, and the implications for users, especially those pushing the boundaries of AI capabilities. They discuss the upcoming shift to pay-as-you-go models and speculate on how this might influence usage patterns and model access. The hosts express concerns over model restrictions for sensitive topics and the potential for government or enterprise use.

Sub-Agent and Loop Functionality in Models

The hosts analyze new features like nested sub-agents, loops, and workflows, questioning their practical utility and target audience. They opine that such features seem aimed at large-scale enterprises or government agencies rather than individual developers or small companies, and they critique the utility and safety of such primitive primitives.

Market Dynamics and Model Usage Trends

The discussion covers trends in API model usage, noting a significant rise in Chinese and other non-US models through OpenRouter. The hosts debate the quality, geopolitics, and cost of using different models, ultimately favoring the best available models regardless of origin, especially for coding tasks. They consider the future potential of open-source models matching closed models in quality.

Advancements in Model Benchmarking and Evaluation

A new coding evaluation from Cognition is introduced, which aims to replicate real-world coding tasks based on open-source projects. The hosts critique its methodology and skepticism about whether it truly measures model quality, especially given the large score gaps between models like Opus 4.8 and GPT-5-5, favoring more rigorous and practical benchmarks.

Market and Product Strategy Insights

The hosts speculate on Anthropic and OpenAI's strategic moves, such as delayed model releases and feature rollouts, and discuss the potential for tiered pricing plans including a hypothetical $2,000/month offering. They also express concerns over the utility and safety of large models and question whether model features are driven by genuine user needs or market theatrics.

Conclusion and Future Outlook

The episode wraps with reflections on the current state of AI model development, their practical applications, and the market dynamics driving feature proliferation. The hosts emphasize that much of the advanced functionality seems aimed at large enterprises and speculate about the societal and security implications of rapidly advancing AI capabilities.

Show notes

Aaron & Ian cover the long-awaited release of Fable 5 & Mythos 5 - does it live up to the hype? Is it worth the cost? Why can't Ian get access to Mythos?

Plus Claude adds nested subagents, Chinese models on the rise, and more