After Outages, Amazon To Make Senior Engineers Sign Off On AI-Assisted Changes (ft.com)
- Reference: 0180948644
- News link: https://slashdot.org/story/26/03/10/212214/after-outages-amazon-to-make-senior-engineers-sign-off-on-ai-assisted-changes
- Source link: https://www.ft.com/content/7cab4ec7-4712-4137-b602-119a44f771de
> Amazon's ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a "deep dive" into a spate of outages, [1]including incidents tied to the use of AI coding tools. The online retail giant said there had been a "trend of incidents" in recent months, characterized by a "high blast radius" and "Gen-AI assisted changes" among other factors, according to a briefing note for the meeting seen by the FT. Under "contributing factors" the note included "novel GenAI usage for which best practices and safeguards are not yet fully established."
>
> "Folks, as you likely know, the availability of the site and related infrastructure has not been good recently," Dave Treadwell, a senior vice-president at the group, told employees in an email, also seen by the FT. The note ahead of Tuesday's meeting did not specify which particular incidents the group planned to discuss. [...] Treadwell, a former Microsoft engineering executive, told employees that Amazon would focus its weekly "This Week in Stores Tech" (TWiST) meeting on a "deep dive into some of the issues that got us here as well as some short immediate term initiatives" the group hopes will limit future outages.
>
> He asked staff to attend the meeting, which is normally optional. Junior and mid-level engineers will now [2]require more senior engineers to sign off any AI-assisted changes , Treadwell added. Amazon said the review of website availability was "part of normal business" and it aims for continual improvement. "TWiST is our regular weekly operations meeting with a specific group of retail technology leaders and teams where we review operational performance across our store," the company said.
[1] https://slashdot.org/story/26/02/22/0650216/amazon-disputes-report-an-aws-service-was-taken-down-by-its-ai-coding-bot
[2] https://arstechnica.com/ai/2026/03/after-outages-amazon-to-make-senior-engineers-sign-off-on-ai-assisted-changes/
This is what happens (Score:2)
When you believe your own marketing too much. Somewhat refreshing to see a company that big acknowledge it's not all there yet.
Cost/benefit (Score:2)
I've seen estimates of the cost of the most recent AWS outage of hundreds of millions of dollars. It's going to be a very long time before the benefits of AI in Amazon's operations outweigh the costs.
Re: (Score:2)
Yeah I can only imagine how much that 2-3 hour outage at amazon's ecommerce site cost them last week. I'm sure a lot orders got put though after the outage ended, but they probably lost out on a ton of impulse shopping, where people later on never came back to purchase.
Too much to ask? (Score:2)
Is it too much to ask for a tiny little lovenote somewhere acknowledging that the PHB that allowed this to happen in the first place is enjoying a severance package?
In what world do you let a newguy do anything important...let alone a newguy that doesn't even breathe.
Re: (Score:2)
Exactly I would assume that anything produced by junior and mid level engineers AI or not was getting a sign off by someone more senior and tested before getting pushed to prod on an environment as large and money producing as Amazon. I guarantee you that heads rolled after that 2-3hr outage they had last week, of course it probably wasn't the people that should have been the gate keepers to production.
Basic rule (Score:2)
Never release anything to production without strict, rigorous, in-depth testing, regardless of the tools used to create it
Re: (Score:2)
Some years ago, marketing updated the old tech phrase from "eat your own dog food" to "drink your own champagne."