News: 1597760829

Just scan the first 500 lines and count all goto's.

Certain uses of goto are a normal and accepted coding practise in the Linux kenel. They use return values and "goto cleanup" to get something like C++ exceptions.

While I get the problem, donating a read-write NTFS module to Linux is a good thing, isn't it? Paragon devs are not kernel devs, but that's hardly their fault. I suspect 20 years ago, someone would have said thanks, jumped on it and cleaned it up for inclusion.

Couldn't agree more - like you say there would have been a time that the community would have been grateful and would have jumped on this to get it included.

Happy days!!

Have to agree here... Although it may have been useful for Paragon to have engaged with whoever has contributed filesystem drivers to the kernel before and asked "how do we do this best?"

That's how I got started contributing to open source efforts... Rather start off on a good note as a newbie than having your Nomex pants scorched.

There are quite a few things to consider when you take on the maintainership of the code, which is basically what is happening and, before it gets to a code review you have to be sure that the legal aspects are covered: one of the reasons why there are so few NTFS drivers is because Microsoft has kept the file system essentially private.

Then there is is the code review. File systems are not new and there might, by now, be a template or at least accepted best practice for their drivers. It's happened before that code dumps were received with "thanks, but no thanks" because there was more work involved in understanding the code than writing it from scratch.

Not really.

Let me give you an analogy.

I've got an old banger of a car. It still works. So I leave it in your garden one day with a note saying it's a gift to you.

The maintenance, updating, API-conversion and upheaval that happens in the Linux codebase is huge. So a dump-and-run is really an obligation, not unlike giving someone a pet dog for Christmas when you don't live with them.

That maintenance burden is FAR FAR more difficult than the initial code-drop is. You have to understand all the code, change it often, make sure it stays secure and bug-free, deal with support from users who now have NTFS 5.1 and why are they getting corruption now when they didn't on 5.0, and so on.

Literally, maintaining that code is harder than writing it in the first place. A dump-and-run is a common output from a company that doesn't WANT to maintain it any more, or deal with the user's complaining that it's unmaintained, or has a bug they haven't fixed.

There were several NTFS projects, historically. Hell, even one that emulated enough of the programming interface to use the original Windows DLLs to access the filesystem under Linux. They've all suffered the same fate - they might "work" but nobody maintains them, so they get bugs and get obsolete and don't work for modern versions of the filesystem, so they end up being a dead-weight in the kernel.

This isn't unique to NTFS, or even a particular subsystem of Linux - maintenance burden is the determining factor in a lot of the patch acceptance pipeline. Everyone involved has to understand it. It has to use the common code that it can (so NTFS doesn't have it's own special way of allocating a file or whatever that's different to everyone else). It has to work internally in a similar manner. Quirks have to be ironed out so they are clearly documented. Filesystem detection has to be consistent so it only offers to run disks that it knows it has the support for, and tied in. And so the filesystems maintainers have to be able to review it and not accept any NTFS-only nonsense.

Because Paragon - if history is anything to go by - will not be patching this code in years to come. They'll be dead, gone, forgotten, unsupportive, the one guy who opened the code will move to another company or whatever. That's how *so many* filesystems, drivers, subsystems, etc. die in Linux. Neglect, and nobody able to understand it to take it over, because it's an "oddball".

All the wifi drivers use the central 80211 framework, which was arrived at after dozens of independent and differing implementations that each vendor tried to put in just to run *their* card, and no other. The commonalities were pulled out, modularised, and then people were expected to conform or explain why they couldn't possibly do that. After decades, all the wifi drivers now use pretty much the same infrastructure even if they are radically different in capabilities, and those manufacturer's are long dead and gone.

Code-dumps are the problem here. Dump-and-run is seen as a charitable action, but it's just an obligation on Linux kernel maintainers to justify, fix, debug, handle user support, etc. for it for decades. And if they can't understand it, or it does things it shouldn't do, then they can't do that, and they'll reject it.

20 years ago people did exactly this kind of thing for NTFS. The result after all that time was one central common NTFS driver that's read-only. Because all those other contributors are long-gone and their code was something that "worked" for a brief period but was atrocious to integrate or debug.

Maintenance is king. Especially in an era where security of something like NTFS, and data security of the user, could easily trash or compromise someone's system and people won't be able to understand how or where it came from. You do *NOT* want a code-dump running your filesystem with any kind of useful data. And you certainly don't want to be responsible for someone else's codedump when your users all blame YOU for their data trashing itself on hardware that you don't even have available to yourself and cannot debug on.

This isn't a gift. It's certainly not a unique gift, it's happened dozens of times before. It's an obligation.

... isn't a gift.

...It's an obligation.

Great write up. +1

Summarised long ago by Virgil, ca. 29/19 BC in Aeneid (II, 49)

Timeo Danaos et dona ferentes

O.

'Timeo Danaos et dona ferentes'

Is that the Timeo Danaos who plays for Barca? I didn't know he was with that Dona Ferentes these days!

Except.... Linux is falling behind when it comes to file systems. ZFS is now highly desired by many users, yet Linux (through no particular fault of the devs or Sun - it's a consequence of history) has no easy way to accomodate it. Now someone is offering a decent NTFS implementation and the initial response is, "thanks, but no thanks". This "it's too big for us to accomodate" for NTFS response would also apply to ZFS if that were to magically be donated under a GPL license.

So current criticism of Oracle sounds somewhat hollow; it'd be spurned anyway, even if it was offered with a GPL license.

The response perhaps ought to be "thank you - hang a mo", followed by a call to arms to raise enough dev effort to properly absorb something this big. Otherwise Linux will forever be rejecting large donations that, really, it ought to be able to accept.

ZFS is actually quite usable these days - I have a couple boxes running it on root via ZoL.

Yes I know that ZFS is perfectly usable and very good - it's just not in the kernel mainstream project.

This is a megapatch not for "Linux" (i.e. the entire ecosystem), but for the Linux kernel .

As another commentard already noted below, there are non-kernel solutions for handling NTFS writes which already work quite well, and have been for many years.

Why should there be a "call to arms" if the prospective contributor has put (charitably assuming) minimal effort into preparing the contribution to be usable, and better alternatives already exist?

(also, I'm pretty sure there's kernel-level support for ZFS nowadays)

There is no official kernel level support for ZFS. There is an separate project, from which end users are free to download and install kernel modules to their heart's content, but the Linux kernel doesn't want to, and indeed can't have anything to do with it, due to license incompatibilities.

Why should there be a "call to arms" if the prospective contributor has put (charitably assuming) minimal effort into preparing the contribution to be usable, and better alternatives already exist?

Because it's a gift. So far as I can tell the prospective contributor has nothing to lose if Linux declines to incorporate the patch into the kernel mainstream. Why should they bend over backwards to please the Linux kernel community? Ever heard the phrase, "Never look a gift horse in the mouth?". Better stuff for NTFS may exist - for varying definitions of "better", but that's not the point (and any FUSE based solution isn't exactly going to be very performant).

Linus has already raised concerns about having enough manpower in the Linux kernel project, and it's easy to understand why; everyone needs to make a living, and the munificense of those companies who donate labour can be stretched only so far. If no one is willing to stand up and do extra work, Linux will end up missing out on things.

I wonder if Paragon's thinking was "we're not sure what we actually need to do with this, let's just submit and they'll give us some pointers as to how we need to sort it out".

If that was the case, then I'm sure it's all going according to plan. However, I suspect they were just being lazy and hoping someone else would just do all the heavy lifting.

"I suspect 20 years ago, someone would have said thanks, jumped on it and cleaned it up for inclusion."

In some ways it is nice to see Linux not quite so reliant on handouts. They can focus on quality and maintenance. This is good evidence that the project is reaching critical mass and one day proprietary companies will be unlikely to compete.

I also see Linux as something that shouldn't become a "code dump". With companies feeling that their end-of-life projects that cannot be monetised can just be maintained for free as part of Linux. If paragon did commit to contributing their code earlier, perhaps they would have been welcomed a little more. I see this quite a lot with embedded obsolete ARM hardware that is no longer sold. It bloats out the kernel without benefiting over 99.9999% of users.

But at the same time, the kernel developers should jump on this one. For too long we have been under the tyranny of Microsofts NTFS and fuse has never really been up to speed in the enterprise space. Also it kinda takes the wind from Microsoft's sails if they ever tried to contribute their driver to Linux for control and publicity (and EEE of course).

I don't think Paragon is 'dumping' their code... but I might be mistaken. As said elsewhere in this post thread, a bit of pre-submission engagement could've prevented the horrible flamethrower-at-your-netherparts moment. :-)

It didn't compile. People had to patch the Makefile.

It has out-of-bounds accesses. People had to run it through static analyzers to spot them. That's a potential "trash everyone's data" right there. It's not compliant with any of the kernel policies.

Also, generally, it would be a pull request of a well-maintained and reviewed tree, not a huge patch on a mailing list. You'd ask for it to be put into -staging or even ask for sign-off first, not just dump it in the mailing list.

They've done everything wrong so far. It's a dump. Or they could have just sent an email saying "We have X, what's the proper path to get that into the kernel?" and be put through to the right people, stage it in their trees for a year or two, iron out the bugs, etc.

Instead they throw a mega-patch onto a mailing list, which didn't build and had bugs visible in seconds in it, and then crowed about how expert they were in doing this. There's also NO TESTS of the filesystem - it doesn't use all the existing kernel filesystem testing procedures.

That's not the way to make friends in the kernel community.

Like I said, Lee, they could've done this better.

But not even compiling... that's not nice. I know another vendor who has form with that... Apple. Download any of their open source source tar balls and try to make them compile (on Mac)... good luck!

*sigh*

"let me build our half of the bridge" doesn't sound much like a flame-thrower approach.

What, you didn't read the subtext? ;-)

Linux is too big and too relevant for the maintainers to accept code from everywhere.

When it was Linus coding away, such a donation would probably have been welcomed. However, when half the world runs on Linux, there's little room for experimentation.

After all, we accept crappy vehicles on our sidewalks (hoverboards that burst into flames), yet we highly regulate what flies above our heads.

Paragon actually sells that code to large companies including Android vendors for great sums of money. Nobody is dumping anything.

Let's say you have a render farm based on Linux and all your artists use Mac or Windows. You need something guaranteed to work with considerable support. You go to paragon like companies.

Only if you are utterly fearless.

For 'no test suite', read 'any change could break things and no-one would know until after your data was trashed'. For 'out-of-bounds accesses' read 'your data was being slowly trashed and no-one noticed until the source code appeared on the LKML'. There are closed source vendors with quality products. Some of them may not take advantage of technological lock-in for a year or two. Paragon does not appear to be one of them.

I will stick with my 'no source code' == 'no sale' policy thank you.

Yeah, but...

20 years ago real NTFS support would have been worth a ton more, and perhaps justified a lot of effort to add it. Now it's borderline obsolete...

Paragon serves its self interest by contributing a minimally functional yet bloated driver. I'll stick to existing NTFS support thanks.

There has been read/write support for NTFS in linux for years. It's a FUSE based filesystem sure but it works.

I follow paragon for years and they did Windows community great favours which also served Linux dual booters.

Thing is, they come from a different culture, development style. I am glad kernel developers didn't go too much harsh at them.

I too have been using ntfs-3g for years. Hasn't give me any issues.