Systems Approach If, like me, you were a computer-science graduate student who cut your teeth on Berkeley Unix – complete with the first open-source implementation of TCP/IP – you know Section 8 as the cryptic System Maintenance Commands section of the Unix User's Manual.

It was obvious, to me, that this concluding section warranted a closer look because the introduction warned: "Information in this section is not of great interest to most users." Judging by my taste in research problems over the years, reading Section 8 turned out to be a pretty good investment.

But before getting to [1]Section 8 , you first learned about [2]the rest of Unix , where you discovered how empowering it is to be able to build new internet applications. Anyone interested in how targeted investments in open-source software, coupled with affordable hardware, can spur innovation should study the role of the Berkeley Software Distribution (BSD) in the success of the internet.

[3]

It's easy to assume the internet as we know it today was inevitable, but at the time BSD Unix happened it was not at all clear that the incumbent telcos could be disrupted. We've commented on the power of APIs many times, but the impact of the Socket API (Section 2) on enabling innovation on top of the internet cannot be overstated. With that stable fixed point in the architecture, a thousand flowers bloomed… and we have, thankfully, moved well beyond the telco vision of B-ISDN.

[4]

[5]

Section 8 was the second half of the story. In addition to describing how to shutdown and boot a system, it defined the process for managing long-running daemon processes, the Unix equivalent of today's microservices. If you had responsibility for configuring and managing system services on your department's server, which came with superuser privilege, you needed not only to know how to program Unix, you also needed to understand the ins and outs of operating Unix.

As a grad student, the lessons I learned while being responsible for [6]sendmail(8) on a live multi-user system were immeasurable. Every mistake instantly sent the faculty into the hallway looking for the responsible idiot. (In my defense, this was at a time when email addresses contained % and ! operators in addition to @, and their precedence was not well-defined.)

The lessons I learned while being responsible for sendmail on a live multi-user system were immeasurable. Every mistake instantly sent the faculty into the hallway looking for the responsible idiot

BSD also provided me with an early lesson in the power of having many eyeballs on the lookout for security vulnerabilities. Looking at the source code for sendmail, for example, revealed a backdoor, whereby one could Telnet to port 25, type the magic "wizard" command, and fork a root shell. So I made my counterparts at Berkeley and other universities aware of that vulnerability by doing exactly that.

Others probably did too, but it was a different time, and the lesson didn't initially take hold. With debugging convenience and a naive sense of community trumping security, the backdoor remained open by default in sendmail until the [7]Morris Worm used it as one of its attack vectors a couple years later.

[8]

Gaining this sort of practical experience is obviously valuable if your plan is to become a system administrator, but it has long been my experience that an opportunity to manage systems that deliver services to actual users is a great source of systems research problems, as well as fertile ground for platform innovations.

My PhD dissertation, born out of frustration with sendmail, turned out to be on naming and addressing; later, real-world experience running a CDN on [9]PlanetLab generated a sequence of systems papers (as Vivek Pai and I reported in a [10]2007 CACM article ); and most recently, our experience operating an edge cloud has led to an appreciation for the [11]state management problem inherent in DevOps. And my experience is far from unique: many of the cloud tools we take for granted today – Kubernetes is a great example – started as someone's response to an operational point of pain.

This all leads me to believe that an open operations platform (as documented in Section 8) is just as important as an open programming platform (as documented in Section 2) for democratizing innovation.

[12]What is this hot, hot thing Magma? An open-source project for building mobile networks, you say

[13]SmartNICs, IPUs, DPUs de-hyped: Why and how cloud giants are offloading work from server CPUs

[14]Everything you wanted to know about modern network congestion control but were perhaps too afraid to ask

[15]Here's an idea: Verification for computer networks as well as chips and code

Would BSD Unix have had the same impact in the 1980s and 1990s if the university computer center had supported it rather than the computer-science department letting its grad students take ownership of the operations problem? We can ask a similar question today. The value of being able to create new cloud applications is abundantly clear, but is there also value in having open access to the tools used to manage and operate the cloud (rather than delegating the latter to the cloud providers)?

To me, the answer is clearly yes. It comes down to the virtuous cycle of solutions being enabled by platforms on the one hand, and platforms being reshaped with the experience of usage on the other. Stable platforms with well-defined APIs surely allow a thousand flowers to bloom, but eventually disruptive refactoring of those platforms is what leads to the next round of innovation.

Stable platforms with well-defined APIs allow a thousand flowers to bloom, but eventually, disruptive refactoring of those platforms is what leads to the next round of innovation

Software-defined networking is a famous example of disruptive refactoring, but it only works if we have sufficiently sophisticated tooling to assemble all the components into a coherent – and manageable – system. Orchestration and lifecycle management have become the dominant operational issues because (a) many smaller parts have to be assembled, and (b) these individual parts are expected to change more frequently. They are essential parts of what we might call the Cloud OS.

Certainly not everyone who writes programs – whether it's running on a personal server or in the cloud – also needs to know how to keep that program running 24/7, but from the perspective of empowering more people to participate in the creation of new systems, the operations platform needs to be kept open and accessible to anyone who wants to invest the time in it.

[16]

Fortunately, there are a plethora of open-source components available today that can be used to operate and lifecycle-manage a cloud. We've documented a roadmap for using them in [17]Edge Cloud Operations: A Systems Approach (a sort of "Section 8" for the Cloud). We're hoping there are still a few people who are just crazy enough to give it a try. ®

Larry Peterson and Bruce Davie are the authors of [18]Computer Networks: A Systems Approach and the related [19]Systems Approach series of books. All their content is open source and available on [20]GitHub . You can find them on [21]Twitter , their writings on [22]Substack , and past The Register columns [23]here .

Get our [24]Tech Resources



[1] https://archive.org/details/smm-4.2bsd/page/n7/mode/2up

[2] http://bitsavers.informatik.uni-stuttgart.de/pdf/att/unix/System_III/

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YgPzPMfN7K@rZ@zTgDOJMgAAAII&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YgPzPMfN7K@rZ@zTgDOJMgAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YgPzPMfN7K@rZ@zTgDOJMgAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[6] https://www.freebsd.org/cgi/man.cgi?query=sendmail&apropos=0&sektion=8&manpath=FreeBSD+14.0-current&arch=default&format=html

[7] https://www.theregister.com/2013/11/04/morris_worm_anniversary/

[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YgPzPMfN7K@rZ@zTgDOJMgAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[9] https://planetlab.cs.princeton.edu/

[10] https://cacm.acm.org/magazines/2007/11/5536-experience-driven-experimental-systems-research/fulltext?mobile=false

[11] https://systemsapproach.substack.com/p/the-source-of-truth-moves-to-the

[12] https://www.theregister.com/2021/12/28/cloud_native_principles_wireless_networks/

[13] https://www.theregister.com/2021/11/24/infrastructure_processing_units/

[14] https://www.theregister.com/2021/10/27/systems_approach_tcp_control/

[15] https://www.theregister.com/2021/09/23/verified_networks/

[16] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YgPzPMfN7K@rZ@zTgDOJMgAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[17] https://ops.systemsapproach.org/

[18] https://book.systemsapproach.org/

[19] https://www.systemsapproach.org/

[20] https://github.com/SystemsApproach

[21] https://twitter.com/SystemsAppr

[22] https://systemsapproach.substack.com/

[23] https://www.theregister.com/Tag/Systems%20Approach

[24] https://whitepapers.theregister.com/