Open source's new mission: To boldly go where no software has gone before
- Reference: 1704711615
- News link: https://www.theregister.co.uk/2024/01/08/open_sources_new_mission/
- Source link:
What comes after open source? Bruce Perens is working on it [1]READ MORE
He should know. Perens co-authored the Open Source Definition document, laying out the principles and purpose by which open source licensing should work. This came out in early 1998, right at the moment the internet burst its banks and became a global flood. The astonishing histories of both have been inextricably intertwined ever since.
It's obviously true that the internet freed software from the physical media it had previously been distributed on. To reach a million users, you no longer had to copy and send out a million spinnable lumps of plastic and metal. You didn't have to be Microsoft to create and broadcast a new operating system, you merely needed to be a [2]Finn with attitude .
Equally importantly, the internet was bidirectional. There was no longer an automatic barrier between the producer and the consumer of software, or between consumers who wanted to become producers themselves. The gatekeepers were, if not gone, no longer protected from competition through economic heft. With open source licensing, moreover, they no longer set the rules of the game.
In return, FOSS gave the internet the tools to grow at its own speed. The early adopters were never going to hang around for Microsoft or IBM to figure things out, they were going to build and share their own servers and infrastructure tools. By the time realization dawned in the boardroom, those rickety early packages had iterated into fully functional, universally deployed, reliable foundations of the new economy. Oops. Double oops: In so doing, they had validated and exemplified how to do FOSS of all kinds. FOSS and the internet reached critical mass, and kept on growing.
[3]
Fans of astrophysics will know what happens to massive objects that keep growing. Gravity overcomes everything and boom – black holes appear. In this case, the critical transition took place in the cloud, the quintessential evolution of FOSS and the internet. Imagine if Larry Page and Sergey Brin, the founders of Google, had gone to Microsoft in 1998 – it's that year again – to ask for a licensing package to build out Google on Windows. Nothing scales like free. And while a lot of big company FOSS usage feeds back to the community, a lot does not. If you don't distribute the code at all, you can lock up your changes too – and cloud services built on FOSS can be a secret as if every byte of code was written inside the event horizon.
[4]
[5]
Or you can be IBM and just wriggle out of things. As Perens bleakly notes, after all these years, the loopholes have been found and exploited, while the inherent problems with FOSS have not been addressed. Equitable distribution of funding, lack of user-focused design, and lack of user awareness of what FOSS can give them, even what it is, means it's time to find a new path.
This arc is inevitable. All revolutions are tamed. The youthful, angry rebels who drove rock and roll, punk, and hip-hop are now establishment-approved fossils in gilded display cabinets. Or dead. Or both. There'll never be another Apache, like there'll never be another Johnny Rotten. There doesn't need to be.
[6]
Traditional FOSS and the internet consumed the world because between the two of them they could hyper-evolve to colonize the new lands of fast comms and ever-cheaper digital devices. There are new horizons for the fleet of foot and itchy of genes, but you have to look at things differently.
[7]Navigating the truth maze in a world of clever machines and cleverer marketers
[8]How the tech toy century has troubled Santa's sack
[9]CLIs are simply wizard at character building. Let's not keep them to ourselves
[10]How to deorbit the Chromebook... and repurpose it for innovators
Perens says that users don't understand the freedoms FOSS gives, and developers don't understand users. He was right on both counts. The software that developers build is the software we talk about – it lives in repos, in source files, in distros. The software users see a multi-headed mutant shape-shifting dragon that swallows up and disgorges data – whatever that is – according to its own rules. Userland isn't built out of software at all, it's built out of data that flies from the screen to the eyes, from the mouth to the microphone. Users know that once inside the dragon, their data can be stolen and abused. They didn't know how to do anything about it, like they don't know why they have to use two calendars and three video conferencing systems. They don't know why the app they've been using for three years just changed beyond recognition.
This is the landscape that needs to be colonized by ideas that help the users. Open source by itself cannot be the cure: a Minotaur's labyrinth costs nothing to enter. What we need is string to find our way out again.
Stop thinking of FOSS software as clever code that lives in git boxes. See it instead as a paving stone or a wall or a destination in the flow of data that actually describes the user's digital world. Make it a condition of code use not just that modifications must be freely available, except when they're not, but that the flow of data must be discoverable to the originator of that data. Where's it coming from? Where's it going? Imagine a traceroute that goes up through the application layer into the data flow between components of what lives up there.
This would extend the inherent transparency of FOSS – something only visible to the highly trained – to the user. The details won't matter to them, but the ability for their device to warn at once if a service or an app or a platform is using a compromised component? With an open, real-time manifest captured available, that becomes possible. As does a "where's my data?" magic button that works independently of service suppliers. It might sound an incredible burden on providers and vendors, but this is 2024.
[11]
The megatonnage of useless data flowing through our infrastructure is incalculable. We know how to define efficient APIs and set rules that can be efficiently served at global scale. Open source has been intrinsic to all that. Now it can blossom into the user-focused, data-centric ecosystem we so badly need. ®
Get our [12]Tech Resources
[1] https://www.theregister.com/2023/12/27/bruce_perens_post_open/
[2] https://www.theregister.com/2019/06/21/linus_torvalds_rant/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZZwqOysy6rWQvqHIi9oABgAAAY0&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZwqOysy6rWQvqHIi9oABgAAAY0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZwqOysy6rWQvqHIi9oABgAAAY0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZwqOysy6rWQvqHIi9oABgAAAY0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2024/01/02/opinion_ai/
[8] https://www.theregister.com/2023/12/25/opinion_tech_toy_century/
[9] https://www.theregister.com/2023/12/18/opinion_column_clis/
[10] https://www.theregister.com/2023/12/11/opinion_column/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZwqOysy6rWQvqHIi9oABgAAAY0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://whitepapers.theregister.com/
Only for a specific type of open source, and only from a certain viewpoint
Again, this is a GPL oriented viewpoint, not a BSD one, but I've banged that drum too many times so..
'Users don't understand the freedoms FOSS gives, and developers don't understand users'
This is simply untrue. Both understand well enough but they just don't care .
For the most part users just want to get on with their life, they do not want to contribute back to the ecosystem, and they don't want to pay money towards it if they can help it. If it cost money, there's commercial closed source software for that. Expecting everyone to buy into your ethos is definitely a case of wanting to have and eat your cake.
Developers also want to get on with their life where that's largely defined as solving the problem for themselves . OpenBSD is probably the epitome of this : it's written by the developers, for the developers, and the fact it's popular elsewhere is a nice bonus.
Polishing the software to truly fit the needs of the end user is a thankless task, and most importantly stops developers from doing what they actually want to do : solve interesting problems, create new code, receive thanks, and in some cases a job that pays money.
Polished open source software would have a magnificent installer, a great interface, all the functions would work, it'd correctly interoperate with almost all window managers/compositors, and be properly cross platform. New functions would not be added until the existing ones were working and stable. Product direction would not be driven by large commercial companies, smaller developers and platforms would be catered for. Ooh look, a flying pig.
Having said that, it's impressive what is available even given the disadvantages. I'm currently trying to use FreeBSD as my main platform to move away from Windows, including for gaming. This has involved :
Having to switch binary packages from quarterly to head because 32 bit wine 8 was only unbroken on 23rd December, and the new package hasn't been built yet for quarterly. This is in the handbook, but it's non obvious, and also not a particularly good default from the point of view to impact to users
Setting a mildly opaque environment variable to prevent old OpenGL games (Quake 2) from exploding with shock at what's provided by a modern OpenGL system (although I don't think it would do that under real Windows : that's on my todo list to check)
Having to make a number of config changes and udev rules so an XBox360 controller is recognised and usable by non root users, and then hopefully works in Wine.
Probably other issues, where certain quite modern 3D games are abending with opaque error messages.
All of this is incredibly user hostile. There are various third party configuration programs I'll be using to provide a leg up, but in an ideal world none of this would be necessary.
Given all that, however, it actually works for free, in a number of cases, without any paid for components being required.
Re: Only for a specific type of open source, and only from a certain viewpoint
I don't entirely disagree, but
> Polished open source software would have a magnificent installer, a great interface, all the functions would work, it'd correctly interoperate with almost all window managers/compositors, and be properly cross platform. New functions would not be added until the existing ones were working and stable ... Ooh look, a flying pig.
Sure, but you might say exactly the same of closed-source software. Setting the bar a wee bit high maybe?
Re: Only for a specific type of open source, and only from a certain viewpoint
You do have a point, but yes that's the ideal for closed source software and that (supposedly) lives or dies on providing for the user's needs.
The problem is perfect open source software needs to provide everything closed source software does *and* maintain the ideals of open source software.
This presents the question that if (as you say) the bar is a bit high for that, then perhaps helping the end user is not in fact the end game, there is a sizeable degree of give and take in return for receiving software gratis, which is the situation at the moment.
Re: Only for a specific type of open source, and only from a certain viewpoint
You make several good points but I disagree with this one.
Commercial software makes no attempt to "provide the user's needs". On the contrary, it just mandates requirements for a particular system & configuration which the corporation will support and provides something which may or may not have much relationship to the user's needs. Often conflicting with the requirements specified for other software the user needs to run. And no attempt to make it work (or even, in some cases, let it try to work) on other configurations.
The FOSS I write, and the changes I make to other FOSS, are mostly about working round some issue or constraint. With commercial software there is no chance to do that - just take it or leave it.
Re: Only for a specific type of open source, and only from a certain viewpoint
Whilst it could easily be argued some software has moved from being user centric to business income centric (especially on mobile) that's not the whole story and it's a hobby horse I'm going to avoid riding today.
Commercial software in general absolutely attempts to provide for the users' needs, at the same time as providing a sizeable return. This is complicated by issues such as captive markets, install base, and interoperating with other products, but it fundamentally has to provide the functionality on some level otherwise it doesn't sell!
I've bought and contributed to both open source and commercial software, but if my intent was to create commercial software I absolutely would restrict the number of supported configurations. This is basic business, both so that the customer can be assured of a reliable product on a known configuration, and also to prevent support and development costs bleeding the business dry.
Much of my recent career is based exactly on providing for customers needs, and then dealing with subsequent technical debt - short term gain for longer term pain. Nevertheless at the time some customised products were released it was to match the customer's needs as precisely as possible for (in general) a very good price for the functionality offered.
This is specifically customised software, various customers are now in the state where their software does not interoperate with newer software or has limitations that don't exist in a more modern product. This isn't because we don't care[1] - it's because we can't make enough money at it! There are options for the customer to move to but it involves a loss of functionality for a more standardised product, a higher cost the customer may be unwilling to pay, or both. As you will know all this has a cost, and if we're doing our job properly we won't sell product we can't make a noticeable profit on. Within the bounds of what the product can do and what the customer is willing to pay, we absolutely do care.
Even with mass market software it often simply isn't true there is no flexibility - it's just there is usually no profit in supporting the missing functionality. Windows for instance has a large number of shims to support poorly written or abandoned software that still has a large install base or are required by a customer providing a large amount of income, because without this compatibility it wouldn't sell or would create huge support costs.
It's not always different in the open source world, but lack of functionality can often be due to either the thing you want to support not being popular enough, or it generating insufficient income or other benefit.
[1] I'm sure some people would say 'well, just make less profit' and 'never accrue technical debt'. If only, it'd make my life a lot easier. However this is a capitalist business out to make money!
Re: Only for a specific type of open source, and only from a certain viewpoint
There is a FreeBSD variant that is good at gaming, it is called PlayStation OS. The BSD license means Sony can keep that non-free. Whether that is a good thing is something the BSD and GNU camps have been arguing about since 30+ years ago/
Re: Only for a specific type of open source, and only from a certain viewpoint
I think the GPL vs BSD argument is a distraction. In practice a Linux distro will include stuff with a wide variety of licences including GPLs 2 & 3 as well as BSD and others. From the point of the typical end user this makes no difference at all. The restrictions of GPL only apply to those distributing modified versions and they are not typical end users.
I do agree to quite an extent with the polish aspect. My thinking about this goes back the Chapter 1 of TMMM and Brooks' discussion of what is and isn't a product explained in this diagram [1]this diagram Some development teams are content to stay in the top left corner but others do indeed produce a product. Those would include LibreOffice and various browsers and email clients. Add to that short list the tools you'd find in, say KDE - Dolphin (file management), Okular (PDF viewer), Kate/KWrite (simple text editor), Gwenview (image viewer), digiKam (image collection management) etc. - and you'll satisfy a lot of users and I'd rate all those as products. I don't think their developers can have considered raising them to that level as a thankless task, nor as one that doesn't involve solving interesting problems.
You also have to remember that there is not a single user audience to be pleased. Kate and vi, for example, are aimed at very different user demands for text editors (and even nvi and vim set out to meet somewhat different sets of expectations). The Kate user will undoubtedly find the vi experience stark and the vi user will find Kate bloated but both are nevertheless right for their respective users.
AFAICS your real gripes were with Wine and that is one I'd marked off as a project rather a product long ago; your comments suggest it still is although I think it has to be conceded that trying to recreate Microsoft's tar pit without getting caught in it will never be easy.
[1] https://images.squarespace-cdn.com/content/v1/4f0ddab6d09a105c7ebcb36d/1387227172150-VOZ6V60CJTXHNI6W9BWL/image-asset.gif?format=2500w
Re: Only for a specific type of open source, and only from a certain viewpoint
It isn't a distraction because the ethos is different : whilst BSD people would like contributions to the community and source it's a nice to have, rather than a central plank of GPL like licences.
I agree that vi and Kate (which I'm unfamiliar with, but can see it's the KDE editor) are different and that it's important to satisfy your target user, not everyone. Nevertheless it's difficult to argue that various mass market software (say the early releases of Libre/Open office before they improved it) were in general not right for the average user unless the average user actually enjoyed writing only a letter and being careful which fonts and layout they used so the software didn't crash.
The reason I used wine was partly because I've recently used it, and also because it's an illustration of what's required to satisfy users that don't know much.
Updating binary packages to fix i386 wine : that's an organisation, policy, and testing issue more than a technical problem. It's not a wine issue.
Quake 2 : wine architectural issue I'm suspecting - it should *just work* with no fiddling. If that means automatically shims like Windows has, so be it.
XBox controller : Partly a 'FreeBSD isn't really aimed at being a great desktop or gaming platform' and partly insufficient resource to make it easy for end users. Maybe GhostBSD, which explicitly is a desktop distribution, is better..
Modern game crashing : unequivocally a wine issue. Debugging issues is a huge pain.
I think trying to weasel out of responsibilities with 'well, it's a project, not a product' is terribly convenient
"but that the flow of data must be discoverable to the originator of that data. Where's it coming from? Where's it going?"
Increasingly FOSS software's USP should be that the answer to the latter question can be "Nowhere, you can keep it local".
What but not why...
As is common for technical discussions, this tends towards the what "we" want to happen ("data flows must be freely visible") and not why.
There's a belief that if only we describe this clearly enough, users will start to care. Why should they? The vast majority of people who go on the internet do so to perform a task "I want to buy a gift for my Mum", not with some deep concern over how it happens. Ideally it should happen with an absolute minimum of fuss and intervention - and there is absolutely no reason for an end user to want to see the complex mechanisms required to make that happen. The best technology being invisible.
The counter argument is often "ah but security" - nope, the average user being able to see code does not improve security, the average number of professional eyes on a given open source project is barely above the one who originally wrote it, and above all if your goal is security (in the legal safety sense) then a vague ability to inspect code is the least robust and practical method to deliver it possible.
FOSS absolutely serves a purpose, but this is not it. It also (as another poster raised and promptly got downvoted for) can be wildly abused as a mechanism for denying the reward for work done.
If, instead of the what, we consider the why (which might reasonably include "we want to enable and positively encourage the free creation of new and better technology for the public good"), then a lot of the gatekeeping, semantic discussion and entirely technically focussed discussions can be seen as either tangential or even antagonistic to the goals. Unfortunately then the hidden politics behind many FOSS debates quickly manifests - the point being not the why, but an attempt to dictate how other people should behave.
Re: What but not why...
the point being not the why, but an attempt to dictate how other people should behave
That is true for all Internet discussions - neither more or less relevant for software than for any other topic (model trains, home baking, legal systems, human rights, etc). Nothing to do with FOSS.
Re: an attempt to dictate how other people should behave.
Perhaps it is only me, but I get dictated to far more by Microsoft - as a result of those who insist I must use their software, than I ever have by anything FOSS.
Re: an attempt to dictate how other people should behave.
I guess you've never been on the receiving end of a long rant of why the key IP for your business should be open sourced, or had users refuse to work with you because they can't see a copy of your internal code. Or users start a project to clone your product simply because you won't give it to them for free.
Old Visions, New Means
Maybe it is time to revisit the original underlying vision of the Xanadu Project, The large scale pattern processing underlying "AI" could be structured to support the mechanics of implementing that vision. But who, these days, would implement such a disruption...
"the user-focused, data-centric ecosystem we so badly need"
The ecosystem we are living in has been defined by companies. Users that are not employees do not have much influence on it, they are not organized and don't have the time to care. Users that are employees follow the diktat of their management, which doesn't care one bit about user-centric. Management is management-centric, employees just need to learn the wierd, illogical steps to give them that shiny graphic at the end of the month.
We need safe-harbour for communities of humans: developers and users.
Open Source software exists either to serve a need or curiosity, either of an individual person or concern or of a community of them. That need or the satisfaction of that curiosity is its sole "mission" and purpose and suffices – no more is required.
However, the Open Source movement – as a philosophy – does have a mission and, as is highlighted by Bruce Perens and this article, that mission deserves new consideration in today's climate of LLMs, where former bastions have been acquired by hostile corporations. That movement's mission has nothing to do with data or software or drawing arbitrary, semantic lines between those two. It has nothing to do with usability or portability or anything technical.
The Open Source movement's concern must focus on protecting the communities that bring the needs and curiosities and gather together – or venture forth as individuals – to build whatever the hell piques their interest or whatever the hell they require.
It must focus on protecting the knowledge and the artefacts from those communities such that they and those that come after may build upon each other's work as Open Source always has done and, without which, Open Source simply could not exist – we wouldn't have any compiler to compile it, any libraries to link it to or an Operating System to run it if it did!
The requirements are crystal clear: what is needed, today, is a return to searchability of knowledge and code and safe harbour for code-bases – i.e. never GitHub, again! We need new places to communicate instead of black-holes like Slack, Reddit and Discord which are not to be trusted and already have strictly finite time-to-live on what's previously written, there. We need improvements to the safety and security of our supply chains: NPM and Cargo and PyPI and the like.
The Open Source movement needs to defend itself from the parasitic predators that pervert the good-faith contributions to build for-profit products. These parasites are invariably "anthropomorphised equity" and they suck the blood of living, thriving organisms nurtured by bleeding and breathing people in order to make their own lines go up – that is the antithesis of "Open Source" AND of "community"! It needs to enable its communities to build what they want to build or need to have – serving users and doing whatever one does with "data" is inevitable and will happen in a more or less successful way, as it always has.
Post-script: I think that GitHub is a good case study. It was once a bastion of the Open Source world and, even today, serves to host the vast, vast majority of Open Source projects – both code and issue trackers, discussions, C.I., wikis and documentation. But, in fact, GitHub is largely a cornerstone of the problem and the reason this debate is interesting in 2024. Before Microsoft weaponized GitHub, all Open Source projects were ruled by a LICENSE and that LICENSE was invariably to be served and stored with the sources. On GitHub, the LICENSE is there but the LICENSE no longer makes everyone equal. Instead, Microsoft have become the pigs in the farm house: they are more equal than others, even though they technically hold only the LICENSE that everyone who fetches the code receives.
Any project that is on GitHub hands to Microsoft perfect visibility not only into every revision of their code-base but also into every single interaction of every other user, developer or viewer of that code and that interaction data is probably far more valuable and far more threatening to the communities about which I'm ranting lyrically, today. Not only that: every fork of every code-base extenuates this problem.
Hosting on GitHub is easy but I wonder if it is not fuelling the machine that will ultimately crush any hope for the future of Open Source as a movement of humans.
Re: We need safe-harbour for communities of humans: developers and users.
EDIT: Obviously, I'm all in favour of building for-profit software and products based on Open Source components but only in good faith and in compliance with the license – that's not what I'm disparaging in my post.
Exploitation
Another article about Open Source failing to mention the other side of the Open Source coin. Exploitation and fostering of inequality.
It's quite telling which side the Open Source promoters are where they don't mention any form of fair compensation to all gullible developers who contributed to it, so that big corporations can make billions off of their work without spending a penny on R&D.