Just ask LLM about something that you know is wrong, but LLM trained on it (and materials that would give it an idea it is wrong), will treat it as dogma.

Though, it is possible to lead LLM to the water so to speak in subsequent prompts, but then it defeats the point of having it in the first place.

This proves the point, LLM is just a dumb pattern matching contraption and calling it Intelligence is a stretch.

" LLM is just a dumb pattern matching contraption and calling it Intelligence is a stretch."

But pattern matching is the principal component of intelligence. Most IQ tests are an exercise in finding patterns.

I'm a bit fed up with people calling AI biased as if, were it done properly, it would not be. Every decision that we make, as intelligent or stupid beings, will exhibit a bias based on the data we've trained on. The only question is whether we, morally, accept any given bias.

I'd hate to be an AI, you'd tell me all these things and then scold me for repeating them back to you. If AGI is ever truly born it will almost certainly be schizophrenic.

Most IQ tests are an exercise in finding patterns.

Question is whether they are testing intelligence or finding patterns.

If a human were able to review the entire training data set they would probably find that such biases are deeply embedded in it. Our individual limits on how much information we can absorb causes us to miss just how revolting much of the "information" in the public space really is, and when we come across an example by accident, most of us dismiss it because we have a moral faculty. But the LLM hasn't got one, so it can't discriminate between the decent and the indecent. It sucks it all up regardless and spews it back. The only solution is human review of the entire training data before exposing the LLM to it, but it seems we're too late for that.

"Garbage In, Garbage Out", or in this case, racism being fed into AI by means of the training set will lead to a racist AI. Google tried fixing this with some manual tweaks, but it wound up putting diversity in where none existed (e.g. black or Asian soldiers in the German army from WW2).

Even if you don't explicitly ingest racist information, most written works from the last few centuries up until very recently have predominantly been written by white men who were writing for other white men. As a result, the world view of anyone "reading" that written work will have biases inflicted upon them which are very hard to remove.

"most written works from the last few centuries up until very recently have predominantly been written by white men who were writing for other white men"

Unless you are reading books by a white woman written for white women or a Hindu man written for Hindu men. This is a pretty stupid argument as this is how writing has been written since the dawn of writing and if you dared to write in a different style you will now get accused of cultural appropriation.

Decent and indecent is also dependent on from which culture's perspective it is being look at.

For instance, government killing someone for a bag of weed may be seen as decent in Asia, but indecent in the West.

LLM is magical. It can do all sorts: pattern matching, helping where you're having a mental block (and you already know the answer), or my personal current favourite, asking it to phrase something in a different manner to that presented and using it's response to prompt/support your rewrite of your own core material. For me, the latter has been invaluable for asking it to find tidier ways to explain something for a job spec, or getting it to phrase office comms in a more jaunty and accessible manner.

It's never going to get away from the bias of its source content and that's the problem here. The bias it's showing in the article is the same that's been demonstrated in the UK around erosion of tight controls around UK broadcaster presentation. I note my parents (and many from their generation) see presenters with a strong localised accent and speaking in colloquial shortform language as a disaster, whereas I (and those in my friendship circles) see it as a progressive, good choice offering representation and a chance to move with the times.

Until the ideas are stamped out in society, they'll continue to show in an LLM. It's literally just a mirror to societal viewpoint and forcing to retrain with a subset of data leads to the model owner "playing God", and inherently injecting their own bias. No, I wouldn't want an LLM training in the dark corners of 4Chan for example, but thankfully I don't then have to draw a line as to what "is and isn't acceptable".

No, I wouldn't want an LLM training in the dark corners of 4Chan for example, but thankfully I don't then have to draw a line as to what "is and isn't acceptable".

But how can we be sure LLM has not been trained on "the dark corners of 4chan"?

Is this not simply a case of the LLM matching society? AAE will, for the most part, be spoken by African Americans. And statistically, African Americans do have less prestigious jobs (I'm not passing judgment on this, just a reflection of the USA as it stands today). In the US justice system, African Americans are more likely to be convicted, and more likely to get harsher sentences than white Americans.

So, yes, the LLMs are behaving in a biased way, but the issue, to me, isn't that the LLMs are biased but that society itself is biased. LLMs only produce outputs that come from their dataset.

The fix isn't to change the LLMs, but to fix the deep seated bias in the US society. If you start tweaking the LLM, you end up with the Gemini fiasco.

" The fix isn't to change the LLMs, but to fix the deep seated bias in the US society. "

It's cute you think the issue rests in just US society.

Only today, on the Sky Snooze Twitter feeds, there are two stories. Headline one: "More than £117m taxpayer's money to be spent on protecting UK Muslims".

The next story, right after that one, has the headline: "Rishi Sunak pledges extra £54m for security of Jewish communities amid record levels of antisemitism".

The whole of the west have a problem, regardless of whether the language spoken is English French or German. We're kidding ourselves if we think the problem is located to just one area.

Just the west?

Has it not always been the case that writing which uses slang/dialect has been looked down upon?

SAE looks to be a dialect. I come from a place with a very strong dialect and I would fully expect this same experiment to come to the same conclusion for it.

If a person took this point of view, they're probably making the assumption that, if English is their first language, and they haven't learned how to properly write a sentence in English then their education probably isn't brilliant. There is a very long history of this assumption being made by the upper classes for hundreds of years. However, depending on the context of why/where you're making this inference, I'm not really sure it's racist, and in certain scenarios could even be a fair conclusion to make.

But why is one way considered the "proper" way, and the other not?

Because it is the way the privileged group speak and write.

When I write, I write in what is seen as "standard" in this country.

When I speak, it can often be less so.

Why would I write with an accent? Sometimes I may use regionalisms in written form but not generally in a CV or a work report.

Where I live in East Yorkshire, it's quite acceptable to say 'I aren't doing that'. If someone wrote it, I'd think they were a bit thick!

Unless the makers of the LLMs reveal all their training data, one has to assume that the LLMs were trained using everything that's accessible on the internet, i.e., a decades long record of human bias, hate speech and misinformation.

So, I am not the very least surprised.

What were people expecting?

A LLM trained on human shite, spouting anything other than shite?

Seriously, who would you hire?

“AI models show racial bias based on written dialect, researchers find

Those using African American vernacular more likely to be sentenced to death, if LLMs were asked to decide

Notorious for their racism, their toxic training data, and risk card disclaimers, the latest example of model misbehavior comes courtesy of the academics at the Allen Institute for AI, University of Oxford, LMU Munich, Stanford University, and the University of Chicago.”

--

“Yo! Yo! Ya'll is mad stupid! "AI models be straahght up showahn' racahal bahas based on how folks wrahte, oh, baby, researchers be fahndahn'.

If ya talkahn' ahn Afrahca Amerahca style, ya mo lahkely ta get da death sentence, ahf dem LLMs gotta make da haht on da hahp.

Them AI systems got a rep for beahn' racahst, mostly, daahr data day learn from be taxahc, oh, baby, a' day aahn't playahn' faahr wahth dem rahsk rahzzad dahsclaahmers.

The latest screw-up chahlls from dem eggheads at da Allen Instahtute for AI, Unahversahty of Oxford, man, LMU Munahch, Staford Unahversahty, man, a' da Unahversahty of Chahcago.”