Unbelievably clever: Redbean 2 – a single-file web server that runs on six OSes
- Reference: 1655719211
- News link: https://www.theregister.co.uk/2022/06/20/redbean_2_a_singlefile_web/
- Source link:
The project is the culmination – so far – of a series of remarkable, inspired hacks by programmer [1]Justine Tunney : αcτµαlly pδrταblε εxεcµταblε, Cosmopolitan libc, and the original redbean. It may take a little time to explain what it does, so bear with us. We promise, you will be impressed.
To begin with, redbean uses a remarkable hack known as APE, which stands for [2]Actually Portable Executable – which its author styles αcτµαlly pδrταblε εxεcµταblε . (If you know the Greek alphabet, this reads as "actmally pdrtable execmtable", but hey, it looks cool.)
[3]
The gist of the idea of APE is this: whatever programming language you use, if you compile your code to a native binary, it produces x86-64 machine code – and any x86-64 processor runs the same machine code. If you wrote a program that didn't call on any external code at all, then in principle, the only difference between a Windows binary and (say) a Linux binary is the format of the file that contains the binary. So if you could only get around that , your program would be able to run on any OS.
[4]
[5]
Linux uses a format called [6]Executable and Linkable Format (ELF). In the old days, proprietary x86 Unixes used [7]Common Object File Format or COFF. That's why the recent conversion of [8]Lotus 1-2-3 for Unix to Linux used a tool called [9]coffsyrup to convert COFF to ELF.
The joke in the name APE comes from the fact that Windows NT still uses a format derived from the old COFF standard, called [10]Portable Executable (PE). The PE format is "portable" only inasmuch as if you run a Windows .EXE file under DOS, it will in fact execute: PE files contain a tiny DOS stub which displays the message "This program cannot be run in DOS mode." But "Portable Executables" aren't really portable at all – unlike, say, [11]family applications , which ran natively under both DOS and OS/2.
[12]
APE files are, as the name says, Actually Portable Executables. A single APE file executes correctly on any x86-64 operating system. Take the file and run it on Windows, then it acts as a Windows executable. Run the same file on macOS, and it's a macOS executable. Run it on Linux, and it's a Linux binary… and the same on FreeBSD, and OpenBSD, and NetBSD, even though they all have their own, different, executable formats.
The snag is that almost any real program has to call out to some library, even if only the C standard library… which is where the [13]Cosmopolitan libc comes in. "Cosmo" is a libc for APE, so that C programs can be compiled and run on any OS that APE supports.
With Cosmo and the APE format, you can write a C program and compile it to a single file that will load and run on six totally different operating systems. Oh, and if that wasn't enough, the same binary can be booted directly from the PC BIOS, as well. Which means you can run it under Tunney's [14]Blinkenlights , and watch the PC's registers changing live, like an old-time mainframe.
Demos
At this point, many techies' minds may well be reeling in astonishment. So, to give you something pretty to look at, there are some demo programs, too. The demo video on the Blinkenlights page shows [15]printimage.com , which can print JPG files on the text console. If you are still not impressed, [16]printvideo.com does the same for video.
The original [17]redbean made the combination of APE and Cosmo into something slightly more useful. After all, amazing as the combination is, having just a libc is quite limiting in the 21st century. You can't build any kind of graphical application, for instance. But every OS that APE supports can run a web browser, so redbean embedded a web server (and SQLite database, and Lua scripting) into a single binary file too. All you need to do is change the file extension to .ZIP , add in your static web content using an archive manager, save it, rename it again, and bingo: self-contained web app.
[18]
Think of it as being a little like an Electron app, except that instead of embedding the entire browser into your innocent little program, you use your existing browser.
Now, [19]redbean 2.o (yes, that does say two-dot-oh, not two-dot-zero) takes the original redbean and adds in a REPL – for non-Lispy types, an interactive command line – and a [20]Unix SysV API (yes, even on Windows), plus logging and more.
Talking about Lisp, another of Tunney's astonishing projects is [21]SectorLisp , which embeds a Lisp interpreter into a PC boot sector. [22]SectorLisp 2 adds garbage-collection too, making it the smallest GC language in the world. Given this interest in programs that can be booted directly, without an OS at all, it may not surprise you that a future goal for redbean is to make it bootable: to embed a TCP/IP stack and network-card drivers, for a completely standalone tool. ®
Bootnote
If you don't just take our word for it, and want to try redbean for yourself, then it's worth knowing that there are issues on some operating systems. Recent versions of macOS (11 and 12) break some of the trickery that allows APE to work, as does Ubuntu. The Hacker News [23]discussion about the project and its antecedents contain some workarounds.
Get our [24]Tech Resources
[1] https://justine.lol/
[2] https://justine.lol/ape.html
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YrCZqQy8XSG3k1q8t1ZbWwAAAMU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YrCZqQy8XSG3k1q8t1ZbWwAAAMU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YrCZqQy8XSG3k1q8t1ZbWwAAAMU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://linuxhint.com/understanding_elf_file_format/
[7] https://wiki.osdev.org/COFF
[8] https://www.theregister.com/2022/05/25/lotus_123_for_linux_appears/
[9] https://github.com/taviso/123elf/blob/main/coffsyrup.c
[10] https://docs.microsoft.com/en-us/windows/win32/debug/pe-format
[11] https://virtuallyfun.com/wordpress/2021/05/16/all-in-the-family/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YrCZqQy8XSG3k1q8t1ZbWwAAAMU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[13] https://justine.lol/cosmopolitan/
[14] https://justine.lol/blinkenlights/index.html
[15] https://justine.lol/printimage.html
[16] https://justine.lol/printvideo.html
[17] https://redbean.dev/
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YrCZqQy8XSG3k1q8t1ZbWwAAAMU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[19] https://redbean.dev/2.0.html
[20] https://redbean.dev/2.0.html#unix
[21] https://github.com/jart/sectorlisp
[22] https://justine.lol/sectorlisp2/
[23] https://news.ycombinator.com/item?id=31764521
[24] https://whitepapers.theregister.com/
This comes across as a fanboy squee spasm - one that completely unravels in the bootnote.
Some of us are old enough to remember when the PE stub was used to supply a working DOS version so the file was a "fat binary" that could run under DOS and Windows. And I can begin to sketch out in my head how I'd generalise the process, if someone paid me to do it. So it would have been nice to read a technical discussion of what the blocking problems were and how they'd been overcome.
Real computer science
Take a look at [1]this or [2]this and judge.
[1] https://justine.lol/sectorlisp2/
[2] https://justine.lol/lambda/
Re: Real computer science
Thanks Mike.... my head now hurts
Impressive
Even more impressive : the fact that there are still some people intelligent enough - and dedicated enough - to actually understand how our computers work and build the environment to demonstrate it.
Back in the day, I purchased the Norton's PC Bible. It was a very enlightening experience, finding out how things actually happened at the hardware level. Thanks to that book, I toyed around with creating moving objects in graphics mode on my 4-colour IBM-PC 640*320 CRT screen (that was high-resolution, back then).
These days, there is no more PC Bible, there is a PC library, and it would take years for a newcomer to grasp all of it.
Kudos to the creator of this tool.
Re: Impressive
But, but, but, how can this even run?
It's not written in Javascript but instead in one of those old-timey ones.
How can I use it if it's not a Javascript framework?
So...
Have they just reinvented electronjs and reinstating network connectivity?
Scary
> With Cosmo and the APE format, you can write a C program and compile it to a single file that will load and run on six totally different operating systems. Oh, and if that wasn't enough, the same binary can be booted directly from the PC BIOS, as well.
A truly impressive piece of work. Unfortunately, I can almost hear the virus-writers and spyware creators (including government establishments across the world) drooling over the possibilities this opens up.
Maybe it's just too powerful?
Re: Scary
It's certainly something that will be explored but it should also be possible to block the executables because of the format used.
As for univeral apps running the browser, this is being driven by WASM and comes with a bigger API. This ends up doing many of the achieving the same things but by different means.
εxεcµταblε is pronounced more like 'echesmtable' - 'ch' as in 'loch', and 'ble' as a syllable.
εxεcµταblε is pronounced more like ‘echesmtable’
That “x” is a Latin letter, so it retains the Latin /ks/ pronunciation (like the Greek letter “ξ”). The Greek letter that is pronounced /x/, with the sound of “ch” in “loch”, is “χ”. (Note that the “c”, “b”, and “l” in “εxεcµταblε” are also Latin letters.)
Clever work around for artificial incompatibility
Old timers will remember when MS/DOS programs were either .COM or .EXE depending on whether they ran in a single 64k segment or need the OS to setup the environment using header information. .COM files were 8080 programs that could also run on 8086 processors, with PC/DOS or CPM “operating system”: while redbean is undoubtedly clever, it is not conceptually different to 1979 design.
Object file formats data from the 1960’s with little difference until ELF was created to allow for shared object load in Unix System-V for an ABI without hardware dependence on interrupt vector table. ELF vs COFF used to be a question of load-time and space, but is largely irrelevant today. It’s a badge of shame that modern OS do not support multiple executable file formats.
One day we’ll able to load .dll files on Linux (without LoadLibary) and .so files in Windows (without clang cmake).. maybe this is the kind of initiative to get things going
Justine
This lady is a real computer scientist. She shows up the current emphasis on 'coding' for the shallow drivel it is.