That was very true 1975 to 1995, a bit less 1996 to 2006. Generally not true at all now. The party is over if you care about power and cooling. An SSD vs HDD for the OS has now more impact for laptop users for new vs 5 year old model.

I'd find it humorous if designers hit an previously-unknown hardware-speed ceiling, and everyone went back to assembler and optimised FORTRAN to achieve further effective-speed increases.

I feel old in that I have spent a lot of time getting C++ code to run faster - execution profilers, custom heap management, that sort of stuff. These days even assembler is a high-level language - the processor is performing all sorts of shenanigans with register renaming, out-of-order execution, speculative execution, contributing to the climate breakdown that may kill us all ...

>I feel old in that I have spent a lot of time getting C++ code to run faster - execution profilers, custom heap management, that sort of stuff. These days even assembler is a high-level language - the processor is performing all sorts of shenanigans with register renaming, out-of-order execution, speculative execution, contributing to the climate breakdown that may kill us all ...

Have you ever looked at your memory map? I find many people don't recognise that a C++ object is basically just a C struct with added subroutines (methods). This means that the memory layout is *forcibly* constrained by the order of member declarations. The compiler *must* add padding between members of different sizes where needed to maintain "natural" memory alignment. So, e.g. on a modern computer, if you first declare a _char_ object followed by a _double_ the compiler must add 7 bytes of unused padding so that the double is naturally aligned.

Unfortunately, at least in the software I've had the pleasure/pain to investigate, developers tend to declare object members in order of perceived functionality rather than memory size. I recall one instance where I went through a fairly complex object, re-ordering declarations so that all methods (presumed to be 64 bit pointers) came first, then all doubles. then all ints/floats, and finally byte-sized members (manually adding dummy members where necessary to maintain element alignment). I was staggered at the performance improvement that one change had on the overall performance of a fairly complex programme. It was in the tens of percents!

I'm not aware if such refactoring is available in modern toolchains. I hope it is, because the compiler itself is handicapped to follow what is laid out in the object definition.

That is something that could be done automatically by a compiler. However while it might "waste" a bit of RAM it would have a negligible effect on performance.

Actually the early C++ compilers for Xenix, MS DOS, Atari, Amiga etc in 1987 by Glockenspiel in Dublin simply converted the source to C and then used a pre-existing C compiler for that platform.

"However while it might "waste" a bit of RAM it would have a negligible effect on performance."

Two words, my friend: "cache lines."

Although normal methods don't appear in the structure so their location doesn't matter. And IIRC there are compilers switches and pragmas to deal with efficient packing and ordering of members because mucking about with some structure will cause ABI issues so the compiler can't do it by default. (These days you even have _Alignof/alignof to query the alignment.)

I remember C++ compilers of that style. Alas I didn't get into C++ until long after "real" c++ compilers came along, and thought it a pity that I didn't pay attention earlier. Seeing how C++ renders down to C is probably an excellent way for C developers to thoroughly understand what C++ is actually all about. Just what is a v-table and why would I iterate over it? Well, here it is!

Possibly the same would be true of Rust. It could be interesting in the C vs Rust debate if Rust could be illustrated as "here's the equivalent C", and see what people make of it.

> That is something that could be done automatically by a compiler. However while it might "waste" a bit of RAM it would have a negligible effect on performance.

It can't, it *MUSTN'T*! Otherwise libraries written on disparate systems would not be able to intercommunicate.

My awareness of this goes back to the days of the Atari ST. People complained that hand-crafted assembler code crashed when accessing a particular C struct. It turned out that the particular struct was defined as something like (simplified)

{byte, b,int i ;}

The 68000 processor could not access odd-numbered memory. The C compiler correctly added a byte padding between *b and *i, but naive assembler implementations tried to access i as (*b)+1.

No, that would be worse. Since late 1980s or early '90s it's been possible to write video games and drivers in a HLL that runs as fast as assembler. Then there is the debugging and maintenance. And Fortran is 1956 technology. Language design and compiler development made that obsolete by the late 1970s. It's just some legacy scientific programs not worth re-writing that has kept Fortran alive.

Actually for cost, reliability and capacity I have Linux on the laptop PCIe SSD, but /var and /home are two partitions on a 2T 2.5″ SATA laptop drive added (one small and the other the balance as the laptop isn't running a humongous web site from /var). A bootable USB stick Linux makes it easy to configure that after the OS is installed on the SSD.

The server would have had to have been spending an awful lot of time in kernel to get that sort of advantage.

Well, it is the network stack. Not exactly the device tree parser O:-)

C.

Something like quad 10Gbe (40gb?) network interfaces with an intel atom synchronizing things.

This research smells of academic theoretical BS to me.

The typical interrupt service routine and kernel context-switch overhead is tiny. Parsing a few lines of json strings in user-space probably has more instructions.

Besides, this, all network hardware in the last 15 years has had buffer coalescing, so interrupts come through at variable rates depending on the load.

You're already well past what a single core can accomplish just in the kernel - and that's despite all the stuff like checksumming that's offloaded to the NIC in such high end devices.

So I would readily believe a 30% speedup for the kind of loads where you have dozens of cores spending 100% of their time in kernel/interrupt processing.

Wonder how this compares to good old interrupt moderation? Sounds like a similar idea - instead of interrupting on every interrupt, wait for either X interrupts before the application is interrupted (heavy traffic) else if Y time has passed since it was last interrupted (light traffic) fire an interrupt.

The patch docs talk about NICs coalescing interrupts anyway, and explain that as the NIC knows nothing about the application's behaviour it's "guessing", and therefore suboptimal.

The mechanism that's been introduced in effect allows an application to give a firm hint as to what degree of coalescing does make sense from the application's point of view, with the side effect that (if judged well) the application will always be asking for data just as the NIC / kernel was beginning to think about raising an interrupt.

Similar or larger gains are achievable in the economy, if the tax code etc were properly reviewed and cleaned up.

Somehow the status quo has been maintained in the countries' governance, as if the actual policies are the wisdom itself. But so much has changed and the changes are accelerating. So maybe the Elon's Dept of Efficiency is not such a bad idea.

I’m confused about how this works in the kennel. Surely your application needs to do the switching between interrupt and polling because (a) only the application knows when it wants to read more data and (b) it is the application that needs to implement the interrupt handler or polling calls

So how is this made “automatic” in the kernel? What am I missing?

Or does this switching only relate to feeding data into the internet buffers for the application to read later? If yes then it still doesn’t answer point (a)

The application doesn't know it wants to read more data till the device driver (in the kernel?) tells it. The application only knows when it wants to send data.

A simplification

Obviously a Web server can't know when network requests happen, and then later responds by sending. A Web browser issues requests (on the whim of the user) which is a small amount sent (unless a form post or upload) and then is waiting for a response. So a server (any kind of protocol) in a data centre (like a telephone exchange) can only roughly know when busy period are, but can't anticipate exactly the instant of demand, or the density of it. It's quite a different network load & traffic to a workstation/Laptop with a single user on a GUI.

Wild guess: the kernel knows whether a process is actively waiting on the socket or busy eating processor cycles. That's probably enough of a clue.

Applications indicate they want to read more data by notifying the operating system via select()/poll()/epoll(). These mean "stop me, then wake me up when more data is available". The details of how to discover whether or not data is available are hidden by the kernel so the application doesn't need to care, it just needs to correctly use the proper API. Note that the poll(), epoll() calls are not a direction to the kernel to use polling instead of interrupts.

Imagine it's a web browser waiting for a response from a server. Simplified, the application opens a socket and sends data to the server and waits for the response by calling poll() on the socket.

When that happens, the kernel kicks in. If there is no data on the socket, the kernel won't reschedule the application. Later, the server responds - an interrupt occurs, the kernel reads the data and then places it in the application's socket buffer. The application is then rescheduled and can immediately read the data as soon as it is resumed.

In the alternative world with polling, the application does not change. Instead of waiting for an interrupt, a timer inside the kernel periodically wakes up and checks for data. If it is there, it follows the same series of steps that it did when it got an interrupt. The application does not have to change.

Some network cards, typically the higher-end ones, already effectively support this with a feature called "interrupt coalescing" where they will wait for their buffers to fill up, or for a timer to expire, before notifying the CPU.

The approach mentioned in the article is likely to be beneficial in high throughput scenarios, but not all. There is a crossover point; if your I/O is frequent and regular, polling is more efficient than interrupts due to the extra interrupt servicing overhead. If the I/O is more patchy, polling may waste CPU cycles through doing polling work which rarely finds available data, and it may also introduce latency as a packet will have to wait for the next polling interval before being serviced. According to the article they're adopting a hybrid approach to switch between polling and interrupts depending on the conditions, which is clever. Tune that right and I'd say this feature will end up being enabled by default for most deployments.

Interrupts are great if your I/O comes and goes at random, relatively infrequent intervals, which might be the case for compute-bound workloads

It's an interesting idea, but I do wonder.

One of the reasons to respond to NIC IRQs is to get that packet in, in memory somewhere, fast, so that more packets arriving on the network can all make their way through the limited buffers within the NIC. If the traffic load is such that the application isn't really keeping up, and one is now polling for available packets, it seems to me that there's the potential for network packets to get dropped. There is a parameter irq_suspend_timeout involved, which seems to be a timeout to ensure that the OS will start paying attention to the NIC if the application has taken too long to ask for more data. The suggestion is that this is tuned by the app developer "to cover the processing of an entire application batch" ; but, what about the NIC's ability to continue to absorb packets in the meantime?

The patch documentation doesn't mention packet loss, drop, etc at all, so I'm presuming that actually that's covered off somehow, hence no need to explain the risk of it.

The thing is, dropped network packets will start having a big impact on the amount of energy consumed by the network itself. It costs quite a lot of power to fire bits down lengths of fibre or UTP, and the energy cost of dropped packets starts being more than a doubling of that power (because there's more network traffic than just re-sending the dropped ones). So that's why I'm interested in whether or not they have got packet dropping covered off somehow.

However, on the whole, a clever idea and well worthwhile!

Tuning Architectures?

Ultimately, if more network traffic is being fired at a host than the host can consume, then the architecture is perhaps wrong, or wrongly scaled. Our networks effectively implement Actor Model systems, which are notable in that a lack of performance gets hidden in increased system latency, because it muddles through the data backlog eventually (or at least, that's the hope). Thus, it's tempting to write off the increased latency as "who cares", and move on. That is often entirely acceptable (which is why all networking and nearly all software kinda works that way).

However, if one adopts a more Communicating Sequential Processes view of networking (think Golang's go routines / channels, but across networks instead, or a http put), this has the trait that if a recipient of data isn't keeping up, the sender knows all about it (send / receive block until the transfer is complete - an execution rendezvous). There's no hiding a lack of performance in buffers in NICs, networks, because there aren't any (not ones that count, anyway). It sounds like a nightmare, but actually it's quite refreshing; inadequate performance is never hidden, and you know for sure what you have to do to address it. However, if you do get the balance of data / processing right, all the "reading" is started just as the next "send" happens, and the intervening data transport shouldn't find a need to buffer or interrupt anything.

This new mechanism brings the opportunity to kinda blend both Actor and CSP. It's "Actor", in that data could build up in buffers, but if an application / system developer did tune their architecture scale just right in relation to processing performance, the packets would just keep rolling in and be consumed immediately with barely an interrupt in sight, as if it were a CSP system but without any explicit network transfer to ensure the synchronisation of sending and receiving.

Of course, achieving that in real life is hard for many applications. There are some where there's constant data rates, e.g. I/Q data streaming from a software defined radio (well, the ADC part of it anyway). This new mechanism paired with the fact that PREEMPT_RT has just become a 1st rank kernel component (another hooray for that!) does some interesting things to the performance that could be achieved.

My experience here is from long ago, but...

It may matter as to how long the polling takes. If control registers/whatever have to be used to enable access to whatever can report that work is ready, you have to include that cost (plus the cost of putting the system back to normal operating state, even if that ends up being done in a quite different context) in the cost of polling.

In a very low-end old-style system, you could have a UART/whatever that makes it status available to the CPU with one access (likely a longer access than to RAM, but still). Polling will be cheap.

On the other end of the scale, perhaps your device is smart enough that you can give it a bunch of memory resources that it can fill up without needing the host CPU to do anything other than handle whatever is ready, when asked. Streaming right along with few needed interrupts and little overhead. I thought ethernet chips started doing this decades ago. What am I missing?

Stuff in the middle might need more host attention, so the tradeoffs will differ.

Given the reported improvements, clearly I'm missing something.

My direct experience was made a bit simpler by having a CPU with only one core, so full/empty flags in memory were enough. Incoming messages went directly into buffers, all within one interrupt (the hardware wasn't quite smart enough to properly clear the "empty" flag, if I recall correctly). The further-processing thread could take buffers that the interrupt code had marked ready, and could return buffers to that code similarly, all without anything waiting. Only when there was no more work to do in the in-memory buffers from the interrupt code would the thread go to sleep (after setting the "wake me up" flag of course!) Another variant for different hardware had a Linux device driver and my code in our hardware use similar conventions between them. These are essentially combinations of interrupts and polling, which is of course what you likely want.