- AI For All
- Posts
- Desperate Times
In partnership with
Hello readers,
Welcome to the AI For All newsletter! Performance improvements in large language models are slowing down, Sam Altman and Dario Amodei say AGI is imminent, and innovation in the field may come from an unusual source. Let’s dive in!
Desperate Times
According to a report from The Information, OpenAI is seeing slower performance gains in its large language models. It’s not just them. Google and Anthropic are also struggling to advance their AI systems. Apparently, OpenAI’s next frontier model, Orion, represents a smaller improvement than the one between GPT-3 and GPT-4 while being far more expensive to run. Orion did not meet OpenAI’s performance expectations, even failing to correctly answer coding questions it hadn’t been trained on. OpenAI formed a team focused on figuring out how to train LLMs as they start “hitting a data wall.”
If you’ve been following this newsletter, then the diminishing returns of LLMs should come as no surprise. Even Ilya Sutskever, an early advocate of scaling, told Reuters that results from scaling up pre-training have plateaued. Furthermore, larger models suffer from data and power shortages and are more likely to succumb to hardware-induced failure. The situation is pretty dire, especially given that the obscene valuation of OpenAI (and the entire AI industry) is based solely on the idea that scaling will lead to AGI.
Some might be tempted to think, “they’ll figure it out.” But remember, today’s AI is based on the discoveries in the Attention is All You Need paper, which was published over seven years ago. There have been no fundamental research breakthroughs since. Now, many AI researchers will cling to LLMs for dear life because it’s all they have, and they will insist that shifting the focus to inference or having AI “think” slower will stop the bleeding. But this misses the bigger picture. The industry must find a new paradigm, along with a sane business model, and reevaluate what it’s even building, which should be a true synthetic intelligence. More on this later.
Despite this damning news, a characteristically evasive Sam Altman implied in an insufferable interview with Y Combinator that AGI could arrive in 2025. He claimed that OpenAI basically knows how to achieve AGI (an obvious bluff that he doesn’t elaborate on), and that the arrival of AGI will be fairly inconsequential. I’m sure it will be. You know why? Because whatever OpenAI ends up peddling as AGI in the future, it won’t actually be AGI. OpenAI defines AGI in whatever way suits them. They’re playing tennis without the net. Saying you achieved time travel doesn’t mean anything if you define time travel as something that isn’t actually time travel.
Sam Altman isn’t the only one making stuff up. On the Lex Fridman Podcast, Anthropic CEO Dario Amodei said that “if you believe the straight line extrapolation, we’ll get there in 2026 or 2027 [referring to human-level AI].” We now know the “straight line extrapolation” shouldn’t be made. When asked about the release date of Claude 3.5 Opus (likely the model they’re having trouble improving), Amodei was evasive.
Remember when Elon Musk claimed that AI capabilities are increasing at a rate of 100x a year? I used to think only Musk would be immature enough to so brazenly abuse AI hype, but now that I see Altman and Amodei doing the same, I realize that this must be desperate times. In Altman’s case, I could point out the sheer fraudalence of this man, the unabashed charlatanry on full display, but I will instead offer an undeserved sympathetic interpretation. As CEO, Altman is in the unenviable position of having to keep OpenAI afloat while its core product stagnates. What else can he say?
One of the primary vehicles for AI hype and the romanticization of the field has been podcasts. It seems that just showing up on a popular podcast garners you good will among internet users, so it’s no surprise that the podcast circuit has become a hotbed for reputation laundering. We need to outgrow this “just asking questions” and “just having a conversation” mindset. The format has been abused by grifters and lunatics to much adulation, due in no small part to the naivete of hosts who simply aren’t qualified to adequately parse two or more hours of bloviating from an ill-chosen guest.
It’s not that there isn’t a place for podcasts in one’s media diet. In fact, I’ll be recommending an episode of the Lex Fridman Podcast later. However, we need to be alert to the limitations of the format. The tech industry’s affair with podcast land is an outgrowth of a certain belief, that information (and its unmoderated flow) is a net good and synonymous with the pursuit of truth. But this belief has only had the effect of polluting the information landscape. Information ≠ truth. Most information is noise.
As we mourn the downfall of LLMs, the question we should be asking ourselves is why. Why was Big Tech so eager to rush out a technology that clearly wasn’t ready for prime time? The answer is the desperate need for growth to appease shareholders, but that can’t last forever. One recent sign of this is Apple warning investors that future products will probably never be as profitable as the iPhone. The tech industry faces a potentially painful reckoning where there are no buzzy hypergrowth markets left to capitalize on. Cloud, crypto/blockchain, VR/metaverse, generative AI, what’s next? Quantum?
The hubris of tech billionaires as of late is probably the most egregious example of the Dunning-Kruger effect in history. They’ve become emboldened by a culture that sees shamelessness (or unapologetic foolhardiness) as a feature, not a bug. They’ve co-opted anti-establishment sentiment to advance their own interests, among which is the construction of their own nation-state in hopes of circumventing any regulation or accountability — their own Neverland where there’s no adults to tell them no, a dystopian plutocracy where every juncture in life requires paying a toll. The next few years will see unprecedented levels of exploitation, opportunism, and value extraction.
So, where do we go from here? Well, we need new approaches to AI. There’s some promise in neurosymbolic AI and hybrid architectures, but perhaps we need to think more outside the box. The problem with building true artificial intelligence (an actual synthetic mind) is that it’s not all that useful to businesses. Selling a conscious mind as a SaaS product would probably be in poor taste. That being said, cyber animism might offer some important insights into building real AI and returning the field to its original philosophical mission: mechanizing and scaling the mind.
A brief summary of the idea: software is not a physical thing, it’s a physical law that animates physical things, hence cyber animism. We don’t exist in the physical world but a dream world. We are virtual, we are software. To build conscious AI, we have to build a machine that models the world and itself virtually (like a brain does). While the roots of this idea extend as far back as Aristotle and Zhuangzi, there is no better speaker on the subject than AI researcher and cognitive scientist Joscha Bach: here’s the short version and the long version (Lex Fridman Podcast). Bach currently works at Liquid AI.
🔥 Rapid Fire
Google DeepMind open sources AlphaFold 3 and releases Gemini iOS app
AI industry rushes to design new benchmarks, LLMs fail at FrontierMath
Research: AI doesn’t understand the world, video models ≠ world models
AI boom to produce millions of tons of e-waste and toxic materials by 2030
Neurology ICU nurse shares ‘terrified’ perspective on AI push in hospitals
US orders TSMC to halt shipments to China of chips used in AI applications
Alibaba Cloud releases Qwen2.5-Coder series, matching GPT-4o model
Microsoft introduces new adapted AI models for industrial use cases
Accenture helps clients with launch of Center for Advanced AI in Kyoto
ServiceNow announces GenAI and governance features on Now Platform
IBM announces Autonomous Security for Cloud across AWS environments
Snowflake introduces new features for deploying GenAI on enterprise data
Japan pledges fresh $65 billion for chips and AI to narrow gap with China
Amazon invests $110 million in university AI research using Trainium chips
Ericsson invests $630 million in Canadian R&D for 6G, AI, and quantum
The Washington Post launches Ask the Post AI search experience
Run Your Sales on Autopilot
Increase the output of your sales team without buying more tools or hiring new SDRs. Onboard Agent Frank, Salesforge’s AI SDR, to fully automate prospecting, personalized outreach and booking meetings while your team focuses on closing deals. Get a personalised email from Agent Frank to see his work in action:
📖 What We’re Reading
“A successful digital future depends on responsible use of AI. The EU AI Act marks a significant step in regulating AI systems and could serve as a blueprint for other jurisdictions. To realize the benefits of AI, organizations need the underlying models and their use to be secure, safe, and trusted. Implementing robust data governance, model-risk, security, and individual-rights management is crucial for responsible AI governance. Together, these pillars create a solid foundation for future digital transformation, and digital trust.”