In 2015 Amazon purchased chip designer Annapurna Labs, [1]remembers IEEE Spectrum , "and proceeded to design CPUs, AI accelerators, servers, and data centers as a vertically-integrated operation."

The article argues that while AMD, Nvidia, and other big-name processor companies may also want to control the full stack (purchasing server, software, and interconnect companies) — Amazon Web Services "got there ahead of most of the competition." (IEEE Spectrum interviews Ali Saidi, technical lead for the AWS Graviton series of CPUs, and Rami Sinno, director of engineering at Annapurna Labs, on "the advantage of vertically-integrated design — and Amazon-scale...")

> Sinno: I was working at Arm, and I was looking for the next adventure, looking at where the industry is heading and what I want my legacy to be. I looked at two things: One is vertically integrated companies, because this is where most of the innovation is — the interesting stuff is happening when you control the full hardware and software stack and deliver directly to customers.

>

> And the second thing is, I realized that machine learning, AI in general, is going to be very, very big. I didn't know exactly which direction it was going to take, but I knew that there is something that is going to be generational, and I wanted to be part of that. I already had that experience prior when I was part of the group that was building the chips that go into the Blackberries; that was a fundamental shift in the industry. That feeling was incredible, to be part of something so big, so fundamental. And I thought, "Okay, I have another chance to be part of something fundamental."

>

> [...] At the end of the day, our responsibility is to deliver complete servers in the data center directly for our customers. And if you think from that perspective, you'll be able to optimize and innovate across the full stack. It might not be at the transistor level or at the substrate level or at the board level. It could be something completely different. It could be purely software. And having that knowledge, having that visibility, will allow the engineers to be significantly more productive and delivery to the customer significantly faster. We're not going to bang our head against the wall to optimize the transistor where three lines of code downstream will solve these problems, right...?

>

> We've had very good luck with recent college grads. Recent college grads, especially the past couple of years, have been absolutely phenomenal. I'm very, very pleased with the way that the education system is graduating the engineers and the computer scientists that are interested in the type of jobs that we have for them.

It's an interesting glimpse into the unique world of designing chips at Amazon.

Graviton technical lead Saidi: I've been here about seven and a half years. When I joined AWS, I joined a secret project at the time. I was told: "We're going to build some Arm servers. Tell no one...

"In chip design, there are many different competing optimization points. You have all of these conflicting requirements, you have cost, you have scheduling, you've got power consumption, you've got size, what DRAM technologies are available and when you're going to intersect them... It ends up being this fun, multifaceted optimization problem to figure out what's the best thing that you can build in a timeframe. And you need to get it right."



[1] https://spectrum.ieee.org/amazon-ai