Netflix reveals massive migration to new mix of microservices, asynchronous workflows and serverless functions
- Reference: 1614668284
- News link: https://www.theregister.co.uk/2021/03/02/netflix_cosmos_serverless_microservices/
- Source link:
As detailed in the streaming firm’s [1]tech blog , the new platform is called “Cosmos” and is the fourth generation of a tool that is used to “process incoming media files from our partners and studios to make them playable on all devices.”
[2]
The previous generation of the platform, called “Reloaded”, has been in use for seven years but was showing its age.
“When Reloaded was designed, we were a small team of developers operating a constrained compute cluster, and focused on one use case: the video/audio processing pipeline,” wrote Netflix senior software engineer Frank San Miguel. “As time passed the number of developers more than tripled, the breadth and depth of our use cases expanded, and our scale increased more than tenfold, the monolithic architecture significantly slowed down the delivery of new features.”
[3]
'Strangler fig' lets the new system grow around the old one and eventually replace it completely
“We could no longer expect everyone to possess the specialized knowledge that was necessary to build and deploy new features. Dealing with production issues became an expensive chore that placed a tax on all developers because infrastructure code was all mixed up with application code. The centralized data model that had served us well when we were a small team became a liability.”
Netflix is famously all-in on cloud and microservices, two technologies suggested as paragons of agility. Yet the quotes above do read rather like an admission the company nonetheless accrued technical debt and has a legacy system to replace.
San Miguel’s post describes Cosmos services as “not a microservice but there are similarities.”
His definition of a microservice is “an API with stateless business logic which is autoscaled based on request load … provides strong contracts with its peers while segregating application data and binary dependencies from other systems.”
DaaS-appearing trick: Netflix teases desktops-as-a-service product [4]READ MORE
“A Cosmos service retains the strong contracts and segregated data/dependencies of a microservice, but adds multi-step workflows and computationally intensive asynchronous serverless functions.” San Miguel wrote.
The developer said that in a typical Cosmos service “clients send requests to a Video encoder service API layer. A set of rules orchestrate workflow steps and a set of serverless functions power domain-specific algorithms. Functions are packaged as Docker images and bring their own media-specific binary dependencies (e.g. Debian packages). They are scaled based on queue size, and may run on tens of thousands of different containers. Requests may take hours or days to complete.”
Cosmos has subsystems, among them one named “Stratum - a serverless layer called for running stateless and computational-intensive functions.”
The post explains a fair bit about how Cosmos prioritises workloads and allocates resources, and how Netflix’s colossal AWS rig is configured to make sure developers and apps have the power they need when they need it.
San Miguel also reveals that Netflix started work on Cosmos in 2018, has used it in production since 2019 and now uses it for around 40 services. The company has chosen the “strangler fig” migration pattern as it “which lets the new system grow around the old one and eventually replace it completely.”
[5]
“2021 will be a big year for Cosmos as we move the majority of work from Reloaded into Cosmos, with more developers and much higher load,” San Miguel concludes. “We plan to evolve the programming model to accommodate new use cases. Our goals are to make Cosmos easier to use, more resilient, faster and more efficient.” ®
Get our [6]Tech Resources
[1] https://netflixtechblog.com/the-netflix-cosmos-platform-35c14d9351ad
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YD4ayYiEPCBNwZkgrBiQjQAAAJQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YD4ayYiEPCBNwZkgrBiQjQAAAJQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2020/07/20/netflix_netfx_desktop_as_a_service/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YD4ayYiEPCBNwZkgrBiQjQAAAJQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://whitepapers.theregister.com/
Re: The problem
They could have reduced the workload and problems with operating at scale by... attracting more use of their platform?
At the scale of Netflix, I always wonder if it'd be cheaper to do it all on prem like Dropbox decided to do a few years ago. For this kind of bursty workload having a $CLOUD autoscale is nice, but for other workloads it's probably cheaper to run your own hardware. Also, most Clouds clobber you on data egress which is bag news for a media streaming platform.
I'd love to see their figures!
Stateless and computational-intensive
You'd hope that you'd persist the state of a computationally-intensive process somehwere or you've just wasted a lot of CPU cycles. Putting the state somewhere else and refusing to look at it doesn't mean it's disappeared. I periodically tidy up my wardrobe and move things around inside - and instances of clothing items might periodically be replaced - but the contents remains basically the same and on the outside it's still a wardrobe.
The adoption of any new technology is an opportunity to spring clean, but I'm always wary of claims that a specific change of design approach has transformed an existing application.
Re: Stateless and computational-intensive
Output is very much not the same thing as state - of course you want to capture the output.
Stateless just means that the output you get depends only on the input you supplied, and not on any /state/ that the service maintained internally (which in turn means that using more instances is much easier, since there is no separate setup to do for each one, and no need to route requests consistently to the same one).
They have invested a lot of resources into FreeBSD which uses Jails rather than Docker (I know not a direct comparison).
I wonder if anyone knows if this recent infrastructure is for a different part of the system or if they have been phasing out FreeBSD?
I think it would have been very cool if they went with Jails rather than Docker, it would have been great to see any presentations they often show to the FreeBSD community on quite how the orchestrated a number of Jails.
Huh. I wonder if they had ever just considered producing and licensing less crappy content. Could have saved money all over the place.