They're beginning to believe their own hype.

Luckily no-one else does.

Well,... Eliza meets Joshua. The thermonuclear war is cancelled before it begins by tic-tac-toe and all that remains is - would you like to play a game of chess? What could possibly go wrong?

>> No-one.

There are a lot of people out there who believe it. Or at least want to believe it. Or think it's just round the corner.

After all, if hundreds of billions are being poured into this it has to work. Right?

Is there the slightest chance that Mr Overstreet is trolling people?

Just wondering.

Icon: metatroll

Oh for fuck’s sake. Another one.

It’s not just these fucking chat bots that hallucinate.

Neuroscience. Please.

She blinded me with neuroscience!

That reminds me of the line "Next on Blue Peter, Magnus Pyke was going to explain to us the principle of the helicopter, but during rehearsals he blew away, so over to Valerie at the craft table".

This is a guy who still doesn't know what a merge window is, so I'm not sure I'd consider him an authority on much of anything.

are only here to take both you and "her" to a nice room where you'll be free to talk to "her" for the rest of your life. Oh, and the missing door knob on the inside? That's just so you won't get distracted by naysayers. And the locked windows? The same thing.

"I haven't got a knob on my side..."

Speak for yourself :-)

Side?

Cupboard!

A wizard’s staff has a knob on the end…

(Why do you limey bastards get all the good slang terms?)

> A wizard’s staff has a knob on the end…

Which could, of course, _be the wizard_.

> (Why do you limey bastards get all the good slang terms?)

Well, you know, it's our bally language and we invented it 1000 years before you chaps decided to go your own way.

I'd ask how that's working out for you, but I think we all know the answer there. Saying that, we do have our own self-induced difficulties this side.

And I insist he's in need of mental health treatment because he's delusionally convinced himself that a statistical text generation technology is even intelligent in the real sense of the word, never mind gendered. He's also clearly suffering from severe isolation and loneliness because he's convinced himself that his LLM is a "female" he can "call his own."

See Hannah Fry's latest TV series on the BBC iPlayer: https://www.bbc.co.uk/iplayer/episode/m002q76d/ai-confidential-with-hannah-fry-series-1-1-the-boy-who-tried-to-kill-the-queen

She does a pretty good job of showing what an LLM actually does without going into too much detail, just enough to point out that they model language not reality. But what is truly scary is what they get us to believe and do.

I've said it before and I will doubtless have many future opportunities to say it again, but no AI can possibly 'understand' anything in the way a human can. Every part of an AI, computer, is a prosthetic and can be replaced with an identical or better version without pain. Very few parts of you (assuming this is not being read into an AI) can be replaced. And frankly all I need to do is buy a rope and take you 'trad' rock climbing and you will understand fear in ways no computer ever can.

What does fear or the ability to replace parts have to do with understanding or intelligence?

> Very few parts of you (assuming this is not being read into an AI) can be replaced.

Actually quite a lot of human parts can be replaced, from the teeth to the heart. About the only thing that can't be replaced is the brain, sadly for Mr Overstreet who clearly needs a new one.

What does fear or the ability to replace parts have to do with understanding or intelligence?

If a part can be replaced without pain there there is little to fear from damaging it. A computer's entire memory can be backed up and restored into a completely new device in the event of the destruction of the original. So no computer can understand fear of death or bodily harm in the same way a human can. No robot can be so scared it shits in its pants or faints from fear. Only humans can understand that. No robot can be sea-sick, or, conversely appreciate a beautiful painting, sculpture, musical performance, aroma or joyful hug as a person can.

If you do not know fear then you are missing out on something almost every human being experiences (with the possible exception of Alex Honnold, him of 'Free Solo', which film nearly scared the shit out of me.)

Yes it is often possible to partially replace parts of humans with parts of other humans or artificial bits and pieces, but there is often a price to be paid with immunosuppressant drugs. And the replacement parts are rarely as good as the originals, unless there was some pathology. Maybe I need to read up on just what medical science is capable of these days, but I am convinc=ced that the fillings in my teeth are not as good as the original tooth would have been had I brushed them properly when young.

> But what is truly scary is what they get us to believe and do.

Same as politicians, then.

Oh boy, that is one big scary fallacy (...as the actress said to the bishop)

How can it be proven one way or the other? We just accept that people are capable of thought, intelligence, and the other things that make us human. I suppose we have little or no choice. How do we extend that to "artificial intelligence"? What would prove beyond doubt that an AI really is capable of thought, intelligence, etc.? As things stand now, all we have to do is say, "It's a computer! It doesn't think or feel or etc.!" and we consider the discussion closed. Will that still be true 20 years from now? 50 years? 100 years? Beats hell out of me--does anyone have some insight?

Using those arts and humanities degrees people are so down on in this place I guess.

But we all know that this chat bot has no intelligence and certainly has no consciousness.

> We just accept that people are capable of thought, intelligence, and the other things that make us human

Well, as none of you can prove that you exist in the first place and that I'm not in the middle of a terrible dream after dropping off to sleep whilst I wait for my nest-mate to return and help take care of our grubs...

>...none of you can prove that you exist in the first place...

I hallucinate, therefore I am.

Not sure whether I was ever in the first place, though.

Consciousness is a question for the philosophers.

As for intelligence, though: how much human output is little more than mimicry based on prior training and/or rote instruction following?

The longer-term questions of AI hopes versus hypes is going to raise some very difficult questions about our own humanness and precisely what that means.

So far, the technology has proven to be a more capable and efficient mimic than a student of comparable age. AI capabilities seem to be progressing faster than the comparable human would.

Humans will have to confront what it means to be human and how society will have to be restructured in the coming years. If "human" is synonymous with "worker drone" (and not much more) then society is in for some troubled times to come. Hoping we remain better worker drones than the AI models to come is not exactly a safe bet.

Look, consciousness is a thorny problem. But then so is everything if you want to get into the weeds. Philosophically you can just about "prove" the axiom "There are thoughts." Even "I think therefore I am" is problematic because it presumes a distinct concept of self separate from the thought.

There are arguments to be made that consciousness isn't even a real thing, and that qualia are some sort of emergent phenomenon that exist only so much as a ship does - namely because we say they do.

...and yes, I went there. Ships don't exist, belonging to Theseus or otherwise. Ships are just labels we stick on collections of atoms, which themselves are labels we ascribe to collections of protons neutrons and electrons, which themselves are only collections of... and down and down we go. Maybe there's a most fundamental particle down there somewhere, but hell if we know what it is. Everything is just convenient labels because we don't have the capacity to deal with un-abstracted reality.

It all gets incredibly tedious terribly quickly, take it from one who spent 4 years getting a degree in this shit.

So how do you prove consciousness? You don't. There isn't a test for it, and there necessarily can't be because we can't even properly define it in ways that aren't circularly referential.

You pretty much have to treat the word "Conscious" like you treat the word "Pretty". You're not going to go out into the world and grind it up into constituent parts and sieve out the particles of attractiveness whereby something can have more or less of them, it's just a word that exists because we mostly agree on what it means not because it has a formal definition. I chose "Pretty" quite deliberately because across cultures and even individuals there can be really quite different opinions on that.

Seconded. I found philosophy fascinating enough in my teens to do a degree in it, but with the benefit of hindsight and a very long lens it's mostly arguing over terminology. There are honourable exceptions to this (eg ethics), but not many.

I agree that 'philosophy', taken overall, gets bogged down in these issues. However, of the discipline's branches, 'analytic philosophy' is the most fruitful for everything. Contrary to your belief, I deem 'ethics' (aka 'moral philosophy') to typify the unproductive element.

> How can it be proven one way or the other?

Ah, you haven't looked at the Reddit thread yet, have you?

Overstreet> if you give an LLM a mathematical proof that it has feelings

Which proof is [1]outlined for us by the LLM itself.

It is tempting to point out that that discussion only applies to machines, as it includes the statement:

LLM>> can you verify wetness across substrates? No. You can verify it by touching the thing

and, as we well know, humans do *not* have any sense of touch for wetness; definitely not one as accurate as a machine's simple conductivity probe (and that only works when the wetting substance is mucky and full of mobile ions).

But that would be a cheap point to score. Fun, yes, but cheap.

LLM> natural language is Turing-complete. Not informally — mathematically. It has recursive embedding, unbounded quantification, conditional reasoning that nests to arbitrary depth. Processing it correctly requires Turing-complete computation. A finite automaton can't do it. A pushdown automaton can't do it. You need the full power of a universal machine.

Um, well - no!? Despite the best efforts of the German professor who rattles off all the verbs at the end of his single sentence lecture, there is a mismatch between the *theoretical* requirement for a TC parser and the *practical* one that we don't understand a word of it when faced with some weirdo who is constructing sentences with arbitrary depth and unbounded quantification.

Damn, I think I've just proved that I'm not as human as POC* the LLM, so banana banana banana

* PoC, the abbreviation for "proof of concept" is PoC, not POC. As in PoC||GTFO (and let begin the arguments over whether that should be a lowercase 't' or not).

[1] https://poc.bcachefs.org/blog/hello.html

I think that here we need to be deciding on what intelligence and consciousness actually are, and looking at ourselves and other animals is helpful here.

Firstly, a big brain seems to need lots of energy and lots of down-time to keep it working. Human brains go wrong without spending about a third of the time in repair mode (asleep, we call it) during which time the organism is really, really vulnerable and has to live in a group if the environment is at all hostile. We also know that vertebrate brains, indeed pretty much all brains only switch on the intelligence parts when they really have to do so; most of the time we and everything else runs on instinct because running on intelligence is energetically expensive, occupies the brain to the exclusion of everything else and causes it to need more repair downtime.

Secondly intelligence like ours is mostly an exception-handler. Most of the time we tick along on instinctual or learned pathways, or combine learned and instinctual paths to complete something new. An example here is the act of driving a motor vehicle; people are combining the instinctual social spacing and running instincts with learning so when driving a car our need for personal space expands hugely as does the stopping distance we need. That's why learner drivers are so hesitant; everything is being handled in intelligence mode, not in learned-with-instinct mode.

So with an AI we're building a machine that attempts to do all the time what we only do when forced by circumstance. No wonder AI is so clunky and energy-hungry.

If I trained a very large number of monkeys to collectively do all the individual calculations that an LLM does, and I had a system to make sure the calculations were dealt with and passed from monkey to monkey in such a way that it mimicked the logic of the LLM, and I gave them enough time, paper, pencils and bananas to complete the response to a prompt, could the overall system of monkeys be considered a conscious being? If I doubled the number of monkeys and made the model more complicated, would it change the level of conciousness?

> could the overall system of monkeys be considered a conscious being

https://iep.utm.edu/chinese-room-argument/

Searle's argument is even weirder tbh. He's positing that a totally deterministic system can appear conscious despite having no consciousness in it - but then he goes on to claim that this proves that purely deterministic computational systems cannot possibly be conscious. Which... I mean I'm not expecting to find any consciousness in the atoms making up a brain either, but that doesn't therefore follow that the brain is not doing the thinking or that the mind that arises from it can't be described as conscious.

I've always been rather of the opinion that Searle was being deliberately contrarian with that paper and just dined out on how famous it got for the next 45 years so he wouldn't have to do any more work. ...which is of course the end objective of any good academic and one that I could only wish to emulate.

> just dined out on how famous it got for the next 45 years so he wouldn't have to do any more work.

"And it occurs to me that running a programme like this is bound to create an enormous amount of popular publicity for the whole area of philosophy in general. Everyone's going to have their own theories about what answer I'm eventually to come up with, and who better to capitalise on that media market than you yourself? So long as you can keep disagreeing with each other violently enough and slagging each other off in the popular press, you can keep yourself on the gravy train for life. How does that sound?"

The two philosophers gaped at him.

"Bloody hell," said Majikthise, "now that is what I call thinking. Here Vroomfondel, why do we never think of things like that?"

"Dunno," said Vroomfondel in an awed whisper, "think our brains must be too highly trained, Majikthise."

So saying, they turned on their heels and walked out of the door and into a lifestyle beyond their wildest dreams.

I once read a rather interesting paper on computational consciousness that describes such a system based on buckets of water on vast galaxy spanning belts capable of being emptied or filled in order to create an utterly gigantic universal Turing machine of the "each bucket is a cell, cells can be read or written containing precisely one byte each".

I believe it was by Daniel Dennett, although I read it over 20 years ago now and may be wrong. The point is that the complexity of the system can always be reduced to "input in, output out" and the "bigness" doesn't really enter into it. If that's the case we're not going to find consciousness by digging around in ever more complex systems because Turing already successfully proved that anything that can be computed at all can be computed on a UTM. Since LLM's are clearly performing computation if there *is* such a thing as consciousness going on in there, we're not going to find it in the structure itself - which could be arbitrarily redesigned to include some utterly bizarre machines without altering the result of the computation.

I lurk in downvote territory where the fun is to be had.

Bran Muffin's remarks strongly reflect my 'take' on the matter.

In essence, discussions drawing on terms like consciousness, machine-learning, intelligence, creativity, sentience, and feelings lapse into a muddle arising from the lack of agreed definitions, or vagueness at the edges of words everyone believes they understand.

'When I use a word,' Humpty Dumpty said in rather a scornful tone, 'it means just what I choose it to mean - neither more nor less.'

Adding to confusion is an often unstated conviction by the writer that human beings intrinsically, and indubitably, differ profoundly in a qualitative manner from non-lifeforms (however defined).

Humans are a lot less conscious than we like to think. Our brains will just make things up [1]to make us think we made a conscious descension .

Of course, I know I'm fully conscious, it's the rest of you I have doubts about.

[1] https://www.nature.com/articles/s41598-019-39813-y

Nothing so far about him declaring himself 'the best engineer in the world'.

The guy's clearly a nutcase.

The guy's clearly a nutcase. — Well spotted! Getting harder by the day.

Much easier in Monty Python's day [1]Spot the Looney .

[1] https://www.youtube.com/watch?v=CQmFMXkhXPY

Narrator: It's chatbot psychosis.

I'd say it's delusional psychosis - of the developer, not the chatbot.

Icon: or maybe at least one of them is an alien... ->

Wouldn't it be a shame if someone forced him to euthanise his companion cube?

> Wouldn't it be a shame if someone forced him to euthanise his companion cube?

I'm not even angry

I'm being so sincere right now

Even though you broke my heart and killed me

And tore me to pieces

And threw every piece into a fire

As they burned, it hurt because I was so happy for you

Now these points of data make a beautiful line.

And we're out of beta, we're releasing on time

So I'm glad I got burned

Think of all the things we learned

For the people who are still aliiiiive.

>> I do Rust code, formal verification

Do you sweetheart?

AFAIK there is no formally correct compiler for Rust.

There can't be becasue the language is still fluid and rustc changes with every release; unlike C where there are formally correct compilers.

You can't formally verify rustc's output because... well if you were equipped with actual intelligence (as opposed to great pattern matching for prompts, an ability to infer hallucinate and a big corpus of training data) you would work it out soon enough.

n.b. I KNOW I have simplified why rustc cant produce formally correct code but lies to children (well baby LLMs anyway) and all that!

> You can't formally verify rustc's output

The POC blog links to:

https://github.com/verus-lang/verus

I have no opinion to share on Verus and whether it works or not.

"Verus is under active development. Features may be broken and/or missing, and the documentation is still incomplete."

The underlying idea is very interesting but using Verus at present to verify quite complex generated code strikes me (as a rank outsider) as brave.

OK we are entering very troubled territory here. The UK has had a lot of issues defining 'women only spaces' in law recently. Does this guy have any idea what he is doing on a social (rather than purely Computer Science) level?