Amazon has several internal reports that show how AI is causing the company to overspend on various projects. The**Financial Times reports that the cost overruns reached $1.8 million, and that is just for one project. These mistakes used to be “trivially cheap,” but AI models made them “catastrophically expensive,” especially as token spending drastically increased with the deployment of AI agents.
The biggest blunder, so far, is the $1.8-million bill that came from a failed Claude Sonnet AI deployment, which was supposed to match author details with listings on Amazon, representing an 860% increase over the allocated budget that was only detected some five months after the issue started happening. Other problems that surfaced include a $541,000 additional cost that came from a project building, ironically, a financial auditing tool, and a $134,000 extra expense for a system designed to reduce delivery times in the company’s logistics network.
“As with any new technology, we’re experimenting, learning and improving how we use it, including how we drive cost efficiencies,” Amazon said in an internal presentation, according to FT. “Cherry-picking small, isolated examples where teams are learning from one another and portraying them as business as usual doesn’t reflect how teams across Amazon are using AI.” And even though overspending more than a million dollars on failed AI projects might seem excessive for the average person, the tech giant’s latest quarterly revenue sits at more than $181 billion, meaning these excess AI expenses don’t even account for 0.1% of what it makes in a month.
This is not the first time that AI-related issues have cropped up in Amazon’s workflow. Earlier this year, AWS reported several outages that were driven by AI coding bot blunders, but the company fixed this by limiting the access of AI agents instead of giving them the same permissions as the senior engineers that they’re tied to. It also used to have an internal leaderboard that showed which employees used AI the most, but has since dropped it as spiraling AI costs made them think twice about the policy.
Many tech companies have been pushing their people to use AI, supposedly to increase productivity through tokenmaxxing. However, the Uber CTO said that there is no link between this policy and shipping successful products. And as agents took over and AI providers switched from subscription to per-token models, costs have become so great that companies are using up their annual budgets in a matter of weeks. While this might not be an immediate issue for tech giants like Amazon and Microsoft, it is unsustainable for most other companies out there.
FollowTom's Hardware on Google News, or* add us as a preferred source**, to get our latest news, analysis, & reviews in your feeds.*
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Jowi Morales is a tech enthusiast with years of experience working in the industry. He’s been writing with several tech publications since 2021, where he’s been interested in tech hardware and consumer electronics.
In more and more "AI" meetings, I've been trying to teach people that local models exist and, while not as "good" (quotes intentional), they're actually a great trade off, even for corpo usage when you have the processing spare capacity and it seems like I'm getting somewhere, which is nice.Reply
This being said, until they don't try to reclaim the "compute" in some way, these stupid things will keep happening for any company that wants to explore AI in any capacity.
Heck, even small companies can get really good local models running pretty much anywhere for cheap and get started with hybrid models.
Regards.
I did chuckle when I read this, not gonna lie. At least I'm not the only one sacrificing to train AI. I really love it when it works, but when it doesn't it's some of the most convincing fail I've ever seen. I can't reclaim that time but I can hope it helps make the next generation AI models better. Thankfully the very worst of AI is usually in the gaming space when you're asking it about strategy, builds, or general game hints. Since it scraped data from Reddit, forums, and every corner of the internet it is full of 12 year old kids wisdom on beta versions of games now 10 years old and the AI has to be "trained" to use a little more common sense before creating the most convincing re-write of the source garbage.Reply