... is Spamhaus-like blacklist of hoarding AI IPs. AI companies are not better than spammers.

The load is not negligible in some quiet but important applications e.g. the kind that let you look up information for multiple scenarios, the AI bot's can crash the system or burn through pricing tiers on cloud deployments. This is leaving organisations including public bodies with the bill and having had an AI scrape content without consent. In many cases with published info like that if they asked they could have a csv, JSON or XML without effectively stealing tax money to max out http servers.

They ignore robots.txt, so they'll ignore this too.

The only benefit is as evidence in a later lawsuit.

“I have a big PLEASE DO NOT BURGLE ME AND STEAL MY VALUABLE PROPERTY” sign on my front lawn and STILL people keep burgling me. Perhaps if I make the sign bigger and use a fancier font and surround it with flashing lights they’ll take heed of my preference not to be burgled?”

No, no, obviously the problem is that you are not specific enough in your sign. As this standard suggests, your sign should say 'Okay, you complete sociopaths can take all my biscuits and tea, but my Meghan porn DVDs are completely off limits - that would be beyond the pale, you monsters!' And I'm sure they'll nod, 'Right then, we're not complete monsters, innit?'

"Dear burglars , Please don't steal my stuff... I will be VERY sad if you steal all of my stuff !!!! So don't steal my stuff OK ?"

And expecting all burglars to honor it...

They don't ignore robots.txt the actively use it to find content.

Its like a putting up a sign saying don't burgle these rooms cos thats where the loot is.

AI companies are all sociopaths who would happily grind live puppies and kittens into blood meal if they could make five cents on it (I have not seen any counterexamples yet, though I guess they could exist, but would get no VC funding). They're like Elmo and the Angry Toddler and Zuck and will completely ignore anything that relies on any sort of social contract or basic human decency. I know, 'basic human decency' is extreeeeemely thin and parsimonious even at the best of times, but they couldn't even meet the most meager standards for that, like not spitting on a guy dying in a gutter for being a n00b who needs to git gud.

I think this is pretty f#$ing obvious now, so any standard that actually relies on the other party not being techbro child molestors is a complete waste of time. Well, at least it lets you prove that you gave them the option and they just ignored it? If that's the only reason this exists and you realize that then I guess it will work just fine, because they will ignore it and you can go 'aha'!

Now this is kind of making me nostalgic for the early days of the internet, when so many dumbasses insisted the internet could just route around censorship as damage and corporations or governments couldn't possibly constrain Good White Men (always white men, of course good) who just wanted to post on usenet while all the women and coloreds took care of the trivial stuff.

Actually, sorry to reply to my own comment, but I did remember that DeepMind has been basically good.

It's a goddamn shame they're owned by Google, which is overwhelmingly evil now, but I think they've still managed to maintain (for now) their balance of doing almost all good things for good reasons. So I wanted to not lump them in with the bastards at Meta, OpenAI, MS, Claude, DeepSeek, etc. I do not think they would grind puppies and kittens into bloodmeal until one of Sundar's incompetent nephews (Just Desi Things TM) demanded it, and maybe not even then!

Make it law that unless there's some kind of explicit consent that says you can, then you can't.

Oh, we already did, but nobody cares and nobody enforces it, so we had to invent random nonsense like robots.txt to try to entice them to obey which really just tells them what you DON'T want them to access but in a way that tells them where it is and which is completely optional and which most bots just ignore and which AI bots don't even bother to check.

This is why everything ends up behind paywalls and accounts. Because people think they can just take whatever they see and do whatever they want with it.

Because, honestly, people with enough money CAN just take whatever they see and do whatever they want with it. Justice is of the rich, by the rich, for the rich. If any normal bloke had done the shite Boris, Elmo, Zuck, and the Angry Toddler have, they'd have been in the nick long ago. And no amount of standards will change this.

Thieves don't pay attention to locked doors, private property or no trespassing signs, the only thing they pay attention to are the punishments.

And, currently, there are no punishments.

That's going to change but probably not soon enough.

Who in their right minds is going to consent for AI bots to use their bandwidth and server resources so that they can then ingest it into their LLM and get no benefit from it in return?

I say blocking their IP ranges is entirely appropriate as robots.txt or other flags might get respected by some AI companies but lots of them will just do it anyway. As we have seen they really don't care about copyright, unless its their own intellectual property getting stolen that is.

'AI' technology is rapidly escaping into the wild. Its penetration is actively supported by one faction of 'capital' (i.e. major players in IT), and strongly opposed by another faction (i.e. smug rentiers of information). It's a replay of brash capitalists of the 18/19th centuries introducing mechanisation and being pitted against Luddites; in the present case, each side commands considerable wealth, and neither merits sympathy as being 'downtrodden'.

What's happening shall in retrospect be deemed an inevitable consequence of emerging digital technologies, these combined with the global reach of the Internet. Meanwhile, there will be heated legal tussles and much prattling in legislatures.

Continuing the analogy to times past, the 'hungry and imaginative' IT-related corporations shall trample over the time-expired and complacent neo-Luddites. On the one side, huge amounts of capital will be expended, much of it to be lost, whilst giants and start-ups pursue ideas which include many dead-ends. The other side will wither on its hitherto bountiful vine.

There is a glimmer of hope of mankind, as a whole, benefitting. This arises in part from the likelihood of computational power continuing to increase whilst cost and energy consumption decrease or stabilise. That is, the development and training for 'AI' implementations is set to escape the grip of extant financial powers and fall within the remit of small companies, universities, professional associations, charitable institutions, and Internet-connected amateurs. From these intermediaries shall flow finely-tuned, cut-down in size, packages, these suitable for use on 'domestic-specification' devices. As for all software products, these will reproduce willy-nilly regardless of attempts to package them as proprietary.

We little people need copyright to be enforced as effectively as it is for the music labels where a few seconds of something in the background gets a takedown.

That so many government idiots seem to be keen on going the other way because AI bots shit out rainbows means that the end result is likely to be everybody ignoring copyright entirely. I mean, if I get ripped off and published authors get ripped off, what's the point?

Fuck this AI shit right between the eyeballs.