Any fool can write a language: It takes compilers to save the world
(2022/04/04)
- Reference: 1649060949
- News link: https://www.theregister.co.uk/2022/04/04/compiling_the_future/
- Source link:
Opinion Here's a recipe for happiness. Don't get overexcited by the latest [1]"C is not a language" kerfuffle.
Proper coders have known since its inception that C is as much a glorified library of assembler macros as anything else. Don't sweat it. That business with operating systems being infected by their old C genes, crippling all the new cool Rusts and Swifts? So what? If your code is limited by its OS interactions, you should probably go write a kernel.
There is one place, and one place only, where you should invest your emotional and intellectual energies. Compilers. They saved the world once, and they're about to do it again.
[2]
For most young code-slingers, compilers are slightly mystical interfaces to the data divinities. If things are good, the completion messages flow down the terminal window like a high priest's blessings on the faithful. If you've done bad, the compiler heaps fire and brimstone on you, and you must repent. When judgement day comes, your executable is your reward.
[3]
[4]
Graybeards know compiler technology is much more complex than simple go/no-go code parsing. In the 70-odd years since [5]Grace Hopper brought compilers into existence, every milestone in processor design has been blessed or cursed by compiler advances or the lack of them. Itanium died because Intel couldn't make the compiler fly. Arm prospered because its compilers made its performance available.
Only compilers can unify the market: C wouldn't be more than a footnote if its compiler technology wasn't easy to port to multiple platforms. Conversely, compilers can divide the market. Think of the 1980s with multiple monolithic C compilers trying to lock products into platforms by making porting painful. They weren't very good, they weren't very compatible, life was slower and harder than it needed to be.
Code contributions to GCC no longer have to be assigned to FSF, says compiler body [6]READ MORE
It is hard to overstate the difference that GCC made to that environment. Compilers can be split into three components: a front end, an optimizer, and a back end. The front end takes your code and digests it into a form the optimizer can then arrange according to rules, and the back end translates that output into the right object format for the target system. GCC liberated this structure, so if you wanted to write a new language, you just had to worry about the front end. A new target instruction set, just the back end. A particular cache structure, hit the optimizer. The benefits are shared with everyone.
That's where the true magic of compilerdom lies. You can have the world's most thoughtfully designed language, but if your compiler makes slow code, nobody will care. With GCC, you got the same kit of parts as everyone else and you could concentrate on what makes your language or processor special. When GCC stopped being the GNU C Compiler and became the GNU Compiler Collection, the economics and scope of compilers skyrocketed, and systems innovation followed them into orbit.
[7]The first step to data privacy is admitting you have a problem, Google
[8]File Explorer fiasco: Window to Microsoft's mixed-up motivations
[9]114 billion transistors, one big meh. Apple's M1 Ultra wake-up call
[10]Enterprise IT finds itself in a war zone – with no script
Then LLVM went a step further, and broke compiler technology into a set of libraries, meaning that some of the things compilers do very well, as in optimally managing data structures and parsing complex commands, can be added into databases and browsers and all manner of applications. Sure, languages are cool, but a complete architectural component model is cooler. Do you want magic in a bottle or to badly reinvent the wheel?
This isn't history. This is the future. Ignore what Intel says: Moore's Law is dead. Single program performance is done. The industry is moving to accelerators, task-specific designs around heterogeneous multi-core, complex memory, massively parallel concepts.
[11]
They're all different; the code you write for one won't port to another, or even to the next version. They are domain-specific architectures needing domain specific languages. Which the accelerator makers build and you take what you're given. We're back in the 1980s C compiler hellscape. Only this time, it's like that for all the languages.
It's very well for the C-spiters to say that your computer is not a souped-up PDP-11. But their computer increasingly isn't even von Neumann or Harvard, with nice unified memory space and homogeneous machine code. Their language assumptions are wrong and getting worse.
The good news is we're learning that massively parallel systems have much in common. They have control CPUs looking after memory hierarchies, tiling, security, and power management that do the same jobs whether you're accelerating TensorFlow or 5G.
[12]
If those can be abstracted sensibly, they can be handled in a common way – and indeed, a new outgrowth of LLVM, [13]Multi Level Intermediate Representation (MLIR), is a project doing exactly that, bringing the proven miracle of open-source compilation right back where it's needed. As for the instruction set for the hardware, it'd be great if that too was open source and highly modular with a rich developmental ecosystem.
[14]RISC-V , anyone? The synergies between that and LLVM have been there from the start.
Most of this is still to come, but it's a way forward that has all the power of history behind it and an economic and technical potential for the industry way above any alternative. By all means, enjoy the language wars; they're pleasant, harmless LARPery. Compiling the future, however, is where the real fun's at. ®
Get our [15]Tech Resources
[1] https://www.theregister.com/2022/03/23/c_not_a_language/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2014/10/14/lovelace_day_2014/
[6] https://www.theregister.com/2021/06/02/gcc_fsf_code_copyright/
[7] https://www.theregister.com/2022/03/28/google_data_privacy/
[8] https://www.theregister.com/2022/03/21/file_explorer_fiasco_column/
[9] https://www.theregister.com/2022/03/14/apple_m1_opinion_column/
[10] https://www.theregister.com/2022/03/07/column_ukraine_enterprise/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://mlir.llvm.org/
[14] https://www.theregister.com/2022/04/01/riscv_taking_steps_to_minimize
[15] https://whitepapers.theregister.com/
Proper coders have known since its inception that C is as much a glorified library of assembler macros as anything else. Don't sweat it. That business with operating systems being infected by their old C genes, crippling all the new cool Rusts and Swifts? So what? If your code is limited by its OS interactions, you should probably go write a kernel.
There is one place, and one place only, where you should invest your emotional and intellectual energies. Compilers. They saved the world once, and they're about to do it again.
[2]
For most young code-slingers, compilers are slightly mystical interfaces to the data divinities. If things are good, the completion messages flow down the terminal window like a high priest's blessings on the faithful. If you've done bad, the compiler heaps fire and brimstone on you, and you must repent. When judgement day comes, your executable is your reward.
[3]
[4]
Graybeards know compiler technology is much more complex than simple go/no-go code parsing. In the 70-odd years since [5]Grace Hopper brought compilers into existence, every milestone in processor design has been blessed or cursed by compiler advances or the lack of them. Itanium died because Intel couldn't make the compiler fly. Arm prospered because its compilers made its performance available.
Only compilers can unify the market: C wouldn't be more than a footnote if its compiler technology wasn't easy to port to multiple platforms. Conversely, compilers can divide the market. Think of the 1980s with multiple monolithic C compilers trying to lock products into platforms by making porting painful. They weren't very good, they weren't very compatible, life was slower and harder than it needed to be.
Code contributions to GCC no longer have to be assigned to FSF, says compiler body [6]READ MORE
It is hard to overstate the difference that GCC made to that environment. Compilers can be split into three components: a front end, an optimizer, and a back end. The front end takes your code and digests it into a form the optimizer can then arrange according to rules, and the back end translates that output into the right object format for the target system. GCC liberated this structure, so if you wanted to write a new language, you just had to worry about the front end. A new target instruction set, just the back end. A particular cache structure, hit the optimizer. The benefits are shared with everyone.
That's where the true magic of compilerdom lies. You can have the world's most thoughtfully designed language, but if your compiler makes slow code, nobody will care. With GCC, you got the same kit of parts as everyone else and you could concentrate on what makes your language or processor special. When GCC stopped being the GNU C Compiler and became the GNU Compiler Collection, the economics and scope of compilers skyrocketed, and systems innovation followed them into orbit.
[7]The first step to data privacy is admitting you have a problem, Google
[8]File Explorer fiasco: Window to Microsoft's mixed-up motivations
[9]114 billion transistors, one big meh. Apple's M1 Ultra wake-up call
[10]Enterprise IT finds itself in a war zone – with no script
Then LLVM went a step further, and broke compiler technology into a set of libraries, meaning that some of the things compilers do very well, as in optimally managing data structures and parsing complex commands, can be added into databases and browsers and all manner of applications. Sure, languages are cool, but a complete architectural component model is cooler. Do you want magic in a bottle or to badly reinvent the wheel?
This isn't history. This is the future. Ignore what Intel says: Moore's Law is dead. Single program performance is done. The industry is moving to accelerators, task-specific designs around heterogeneous multi-core, complex memory, massively parallel concepts.
[11]
They're all different; the code you write for one won't port to another, or even to the next version. They are domain-specific architectures needing domain specific languages. Which the accelerator makers build and you take what you're given. We're back in the 1980s C compiler hellscape. Only this time, it's like that for all the languages.
It's very well for the C-spiters to say that your computer is not a souped-up PDP-11. But their computer increasingly isn't even von Neumann or Harvard, with nice unified memory space and homogeneous machine code. Their language assumptions are wrong and getting worse.
The good news is we're learning that massively parallel systems have much in common. They have control CPUs looking after memory hierarchies, tiling, security, and power management that do the same jobs whether you're accelerating TensorFlow or 5G.
[12]
If those can be abstracted sensibly, they can be handled in a common way – and indeed, a new outgrowth of LLVM, [13]Multi Level Intermediate Representation (MLIR), is a project doing exactly that, bringing the proven miracle of open-source compilation right back where it's needed. As for the instruction set for the hardware, it'd be great if that too was open source and highly modular with a rich developmental ecosystem.
[14]RISC-V , anyone? The synergies between that and LLVM have been there from the start.
Most of this is still to come, but it's a way forward that has all the power of history behind it and an economic and technical potential for the industry way above any alternative. By all means, enjoy the language wars; they're pleasant, harmless LARPery. Compiling the future, however, is where the real fun's at. ®
Get our [15]Tech Resources
[1] https://www.theregister.com/2022/03/23/c_not_a_language/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2014/10/14/lovelace_day_2014/
[6] https://www.theregister.com/2021/06/02/gcc_fsf_code_copyright/
[7] https://www.theregister.com/2022/03/28/google_data_privacy/
[8] https://www.theregister.com/2022/03/21/file_explorer_fiasco_column/
[9] https://www.theregister.com/2022/03/14/apple_m1_opinion_column/
[10] https://www.theregister.com/2022/03/07/column_ukraine_enterprise/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YkrB1mwqIG1OtZsyIUDkeAAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://mlir.llvm.org/
[14] https://www.theregister.com/2022/04/01/riscv_taking_steps_to_minimize
[15] https://whitepapers.theregister.com/
GCC vs C++ / ELBRUS
fg_swe
With the mentioned approach; the compiler can also run on ELBRUS, which does not have a gcc compiler. In fact, ELBRUS has a secret instruction set, but that does not stop me generating Sappeur programs for this CPU.
KISS 2: Generic Code
fg_swe
Some people say you need a dedicated generics mechanism, similar to C++ templates.
This is not true. A proper macro processor such as m4 can (for all practical purposes) do the same. Debugging is much easier, as the "instantiated" code can/will exist in a file to be inspected with a standard editor. No insane, cryptic C++ template error messages.
Not my invention, saw it when I worked for D'Assault on CATIA.
I miss a critical note and some figures.
Marco van de Voort
I totally miss any form of criticism i the article. It is just a article about baseless glory story. No analysis of how it actually works out for non corporate 3rd party users. No numbers, not even names of successful independent frontends and how they do/did it. It could have been an old glory story from the days of the GIMPLE introduction that got polished up a little.
I agree with fg_swe, actually in recent years we saw a decline in attempts to use the backends directly, and go to last resort C backend (which often means giving up the hope on a speedy compiler-run-debug cycle). Only corporates with vast manpower can afford to tangle with these beasts.
On GCC the major version transitions are notorious and break your frontend all the time, and you can only get an old version into a Linux distro for so long. LLVM has so much undocumented behaviour (other than look how C/C++ frontend does it, that you don't even get that far. Both teams are notorious for ignoring bug reports and merge requests for issues that don't touch the dominant frontend(s) and/or their corporate sponsors, even for multiple major cycles. So basically if you run into a problem you are fscked.
Performance issues are often papered over with parallel compilation that is also not that easy with a new frontend.
Too Scary, Too Complicated
As with every nontrivial technical endeavor, one should apply the KISS principle. Demonstrate your innovative idea in a way that is as straightforward as possible. You can always perfect it later.
What does this mean ?
Don't use the gcc infrastructure, instead generate C or C++ code and let existing compilers do the optimization and architecture-specific work. Debugging a compiler that emits C++ is way much easier than debugging ASTs and machine code. As an added benefit, your compiler will be able to run on almost any CPU, because almost any CPU has a C++ compiler.
This is what I did here
http://sappeur.ddnss.de/
The innovative aspect is that this language is a memory safe C++ variant. All the goodies of C++ plus single- and multi-threaded memory safety, due to a type system which "knows" of single- and multi-threaded sections of the program.
There exist Eiffel compilers which use a similar concept of emitting memory-safe C.