Discuss AI Here

[cjs]

In fact, I suspect that most AIs have only been trained on text/information available as html cleartext; that is, I've not found one yet that has probed the depth of the photo-imaged manuals on, say, bitsavers. Given the pitfalls of OCR, I suspect that might not happen for a very long time.

I don't see this as a huge issue, given that so many of the books have been OCR'd already. Including everything on bitsavers. (I don't know how many PDFs in the bitsavers archive itself have been OCR'd, but archive.org has them all, has OCR'd them all, and makes all the OCR'd text available on the web as plain text files as well as PDFs with text.)

I still haven't seen anything said other than AIs that don't "know" facts can come up with the same garbage as humans that don't know facts.

You still don't get it. Humans can know facts, even if only a relatively small proportion of the population happens to know any particular facts. Other humans can learn them. LLMs cannot know facts, period. All they can do is use carefully channeled randomisation to spit back text that's similar to what they've been trained on.

I cannot for the life of me understand why, when an LLM exhibits stupid behaviour, you think that it's ok because humans often also exhibit stupid behaviour. Humans also exhibit smart behaviour, and LLMs never do, though they may appear to do so. And LLMs do fool people like you, because that's exactly what they're designed to do.

The humanity or not isn't the problem, it's the statistical reliability of its solutions, which can be plotted against the statistical reliability of its human peers to determine where it falls on any given subject.

Yeah, that's basically the whole problem with the thing. "Do random stuff and see if, statistically it come out better than something else." This is very typical of software folks, and that's why software, unlike bridge building, is not an engineering discipline.

 

[cjs]

If you're worried about an AI taking your job, then the obvious solution to me is to be better-versed than the AI.

Really? So my manager, who has only a very minimal understanding of software development, is going to stick with expensive me rather than a cheap "AI" when he can't tell the difference between the two?

You might not trust an AI over an SME, but plenty of people would.

 

[segaloco]

You still don't get it. Humans can know facts, even if only a relatively small proportion of the population happens to know any particular facts. Other humans can learn them. LLMs cannot know facts, period.

To be pedantic, humans can't know facts either. Humans can know patterns that they've seen unfold again and again and again backed up my statistical measurements gathered by their brains. Humans once knew the "fact" that the sun revolved around the earth. Nothing up until they learned any different required them to know, and the analysis they had done up til that point had landed them on the same incorrect conclusion. Their statistical models had been "trained" on a world that did not conceive of the heliocentric point of view. Get to a point where it matters and the science starts chasing a more accurate picture of things, and suddenly what had been fact for most of humanity fell apart.

Further using that example, pedantically, the Earth is not in a pure isolated revolution around the sun, but rather, the exact center of that revolution, while proximal to the sun, is influenced by the gravitational tug of the rest of the solar system, local galactic neighborhood, what have you.

Facts are only as meaningful as what we back them up with. There are implementations of CPUs that leave out features, that introduce bugs, etc. If a human had been used to working with a particular implementation of an IC that was known to differ from the datasheet, to them, what the datasheet says is not fact, regardless of what conventional wisdom might imply. This is the case with the 16550 UART implementation in the JH7100 RISC-V core, it does not adhere to the 16550 description of a register field that indicates the transmitter holding register has drained. I had to learn this the hard way writing a serial driver for the thing. In this case I discovered a "fact" that is precisely the opposite of what authoritative literature said.

This is not to say the LLM you are playing with has interacted with 8080 implementations that differ in their handling of the carry bit, but just to demonstrate that even authoritative primary sources of supposed unquestionable fact can be wrong because individual situations are prone to entropy. The LLM is wrong about the 8080 and you proved it, so nobody in their right mind should trust that model with 8080 programming. That doesn't, in my mind, make it any different than a human programmer that also doesn't know that but insists that they do. They better have a good explanation for why they disagree with established understanding, whether they are a computer or human.

End of the day, you were lied to by a computer, but we are lied to by people every day, both maliciously and ignorantly, so it doesn't tell me LLMs are completely untenable and should be completely ignored, it tells me they are just another tool to be assessed for a job and applied if they are found to adequately perform that job. Their lack of humanity should be a deciding factor on what human necessitites we allocate to them (land, energy, water) rather than what we will and won't use them for. I think the extreme resource demands of AI are the real matter that needs to be looked at, not how correct or not they are. We can vet humans vs AIs for tasks, that's a lot lower risk than running out of material resources to sustain life because they're going into keeping AI data centers running instead.

 

[cjs]

To be pedantic, humans can't know facts either.

Yeah, ok; I see where you've gone off the rails here, but I don't know how to help you out of it. I think possibly you don't believe that knowledge exists. Maybe try reading the Wikipedia page on it and considering whether an LLM can have justified true belief.

Humans can know patterns that they've seen unfold again and again and again backed up my statistical measurements gathered by their brains.

Yes, and humans can also know that 2 + 2 = 4, which is nothing to do with patterns or statistical measurements, but is a truth. (I am of course assuming some standard definitions there; if you want to quibble about those, use Metamath's idALT as an example instead.)

Humans once knew the "fact" that the sun revolved around the earth.

No, that was never a fact. Surely even you understand that.

If a human had been used to working with a particular implementation of an IC that was known to differ from the datasheet, to them, what the datasheet says is not fact....

You are terribly confused here. In your example what the datasheet says is fact; that IC is simply not the IC referred to by the datasheet.

 

[MrImprovement]

My personal view is that the use of an LLM to write Prolog, then use the Prolog program as a long term "memory" has potential, as seen here: https://shchegrikovich.substack.com/p/use-prolog-to-improve-llms-reasoning

 

[segaloco]

If you really want to get technical, integer math is even just an abstraction. Everything is made of smaller things, so yeah 2 + 2 = 4 because 2 and 4 or abstract values we've invented to represent the idea of absolute quantities. Absoluteness in quantity is very hard to prove, just based on perspective alone. One person may say they have three apples. I say they have 2.95 apples because the stem is missing from one of them. The integral nature of a calculation is for convenience but an integer is an abstraction, not a concrete thing based in some unquestionable reality. It holds very well with how we analyze our universe, but in reality the universe is not required to operate on integer anything. This is why the definition of, for instance, elementary particles keeps changing. We keep finding that things are made up of smaller things, each with their own properties that contribute to the sum of the whole. Sometimes we discover discrepancies the next level down that redefine the specifics of something, even if they don't fundamentally change how those different factors interact at the macro scale. It's kinda like how Newtonian physics equations have long been demonstrated to not be absolute, but still work well for many situations, so well in fact that most folks in a general physics class are still being taught the Newtonian model. It has been demonstrated not to be absolute fact, but humans have long operated on a principle of "good enough". All that's changed here is now computers are being programmed to likewise generate content that is "good enough" based on statistics rather than strictly adhering to one specific human's abstraction of a problem into a set of operations.

Also funny thing about any article written on knowledge. It is by definition going to be biased towards human exceptionalism *because it is written by humans*. To suggest a bunch of amino acids floating around in a pool could eventually lead to a being like us but a bunch of training data floating around in a malleable algorithm couldn't lead to what we consider independent thought feels almost arrogant. Like it's been said up thread and elsewhere, AI is just closer to its primordial soup stage of evolution than we are.

I'll also add I don't like any of it. I'm not saying any of this as an evangelist for generative AI, I dislike it in practice. I'm just advocating for the fact that it is rightfully challenging what we consider "intelligence" because all along our judgement of what intelligence means has been incredibly biased towards human society. Not that we can help it, that's just the nature of our being.

 

[Svenska]

I don't see this as a huge issue, given that so many of the books have been OCR'd already.

Including OCR mistakes. And a lot of old books were either not scanned or printed well enough for OCR to be fully reliable. We cannot even make scanner which capture the image correctly (remember Xerox and JBIG2 a while back?)

Humans can know facts, even if only a relatively small proportion of the population happens to know any particular facts. Other humans can learn them. LLMs cannot know facts, period.

Most facts you know because someone told you. You don't have the ability to personally verify them all. In fact, much knowledge (large parts of history or medicine) is just conjecture based on fitting fragments we found - especially in the past. We send kids to school in order to teach. I don't see why teaching an LLM prevents it from ever knowing facts, but teaching a kid does not. We likely operate on different definitions of "know" here.

Imagine I went up to someone in this forum and had this conversation:

Me: Does 8080 POP AF preserved the carry flag?
Person: Yes.
Me: You're wrong. It does not; here are two separate references to manuals that make it clear it doesn't, because it's loading the previous value from the stack, overwriting the current value of the carry flag.
Person: ...
Me: Does 8080 POP AF preserved the carry flag?
Person: Yes.

Hmm, this discussion feels a lot like that.

Person A: Can AI be useful?​

Person B: No.​

Person A: Here are a few areas where AI already being used successfully. Here are some references of people who claim that AI works well for them. Here are ideas where the current state of AI appears useful within reason.
Person B: ...
Person A: Can AI be useful?
Person B: No.​

​

Was thinking about this while driving to work today.

 

[MissArgent]

Whole thing sets off too many of my "This is a speculative bubble" red flags, particularly with continually being boosted by the most irritating human beings alive. Comes off as more of a "flavor of the week" than a major development.

 

[Chuck(G)]

If you really want to get technical, integer math is even just an abstraction.

Can't wait to read the first edition of the AI-generated Principia Mathematica.

 

[segaloco]

Can't wait to read the first edition of the AI-generated Principia Mathematica.

Yeah, I know I'm getting into "*puff puff* numbers are just like, an illusion man" territory...but these sorts of analyses are better to figure out now than retrofit into this stuff 20 years from now...

 

[Agent Orange]

Can't wait to see the first AI generated Playboy complete with photos and captions.

 

[MrImprovement]

Can't wait to see the first AI generated Playboy complete with photos and captions.

Do you want to read it for the articles?

 

[Bentendo64]

I'm finding this thread entertaining. Thanks everyone.

I do believe the question "can an LLM/AI posses intelligence?" is a philosophical question. I think that's the sticking point as to why cjs and the others are talking past each other. From a naturalistic (scientism) point of view, I do believe one could logically arrive at the conclusion that these programs have the potential to be intelligent. However, removing the naturalist presupposition, the proof comes to an end pretty quickly when you look up the definition of intelligence.

 

[segaloco]

The definition of intelligence as solely defined by the musings of the homo sapiens

 

[cjs]

If you really want to get technical, integer math is even just an abstraction.

Well, yes, in a certain sense. But I don't think it is in the sense you're using.

Absoluteness in quantity is very hard to prove, just based on perspective alone. One person may say they have three apples. I say they have 2.95 apples because the stem is missing from one of them.

I'm at a bit of a loss here. Are you not aware that apples and integers are vastly different things? You are talking about using integers as an abstraction for counting physical objects, but while that can be useful, that doesn't mean that integers are physical objects, or that physical objects are integers.

To understand what I'm talking about, you need to stop trying to correlate mathematical concepts with physical objects and understand that there are mathematical ideas per se that one can manipulate with logic, regardless of whether the real world is capable of "living up" to these or not. And those are the answers I'm looking for from an intelligent analysis.

Is the Intel 8080 manual wrong when it says that the CPU always clears the carry when I execute an AND instruction? Well, yes, because it doesn't always. If I run the CPU on 3.5 V rather than the required 5 V, perhaps it malfunctions. But any intelligent being would know that going down that road of argument is counterproductive; we want to talk about things as they should work, and consider the above a failure of use, and bugs as a failure of design, rather than going for the nihilistic, "the world is random; we can't try to do anything" approach.

The integral nature of a calculation is for convenience....

No. It may be convenient for you to use integers in certain situations as a good enough approximation of what you need to calculate, but integers in math are not a "convenience"; they are a real thing that lives in logic.

...but an integer is an abstraction, not a concrete thing based in some unquestionable reality.

I'm not sure where or how the "reality" comes into this or not. Integers are a concept, based in certain rules, and if you follow these rules you can come up with valid results. That is unquestionable, because the rules are there, and they have no necessary connection with the physical world. You may choose to apply these ideas to the physical world, but do not mistake your application to mean that integers now must match and have all the failings of the physical world.

It holds very well with how we analyze our universe, but in reality the universe is not required to operate on integer anything.

Yes, exactly. But how does this matter when I am dealing with mathematical and logical concepts, and ChatGPT is failing to do so?

This would be a big part of my expectation of "intelligence": it can understand abstract systems and work within those rules. Which LLMs clearly can't.

Most facts you know because someone told you. You don't have the ability to personally verify them all.

When I'm doing logic, there's no need to verify anything. I simply look at the rules, work within those rules, and come up with a result. And that's where the LLMs entirely fail. You tell them, "assume X is true, tell me about Y" and they reply with, "You are right, X is true," and then give you something that by logic only works if X is false.

The universe is a big complex place, and we often have false beliefs. But the ability to work out valid conclusions, whether they're true or false, is a huge part of why we've come as far as we have. And LLMs, apparently, have no way to do that. You can give them all the facts in the world, but if they've been trained on more "the sun goes around the earth" text than "the earth goes around the sun" text, they'll come out with the former. (And even if they've been trained on the latter, they will still occasionally come out with the former.)

I don't see why teaching an LLM prevents it from ever knowing facts, but teaching a kid does not.

You can train a child to follow rules. You cannot train an LLM to follow rules.

 

[Svenska]

When I'm doing logic, there's no need to verify anything.

Formal logic, and only up to the incompleness theorems, yes. But all sufficiently complex systems are either inconsistent or incomplete.
Even worse, the real (human) world operates on far weaker assumptions. Religion is far from formally sound, yet a major part of human life. The whole world of sociology and psychology deals with inconsistency in human beings... with varying degrees of success.

But the ability to work out valid conclusions, whether they're true or false, is a huge part of why we've come as far as we have. And LLMs, apparently, have no way to do that.

Current-generation LLMs don't have the ability, that is true. But why do you conclude that they will never have it?

As a side-note: Basic image generators or autocompletion engines do not need the ability to proof logic theorems to be useful in the first place.

You can train a child to follow rules. You cannot train an LLM to follow rules.

LLMs execute on Turing machines, operating with a well-defined set of rules, consuming a well-defined set of inputs and produce a well-defined set of outputs. That makes them computer programs, by definition. If you can train a computer to follow rules, you can also train an LLM running on that computer to follow rules.

Verifying whether the LLM - or the child, for that matter - will follow the taught rules consistently is not as easy. Given that LLMs are deterministic, they have the advantage here. Of course, in LLMs, we separate learning (training) and using (inference) stages, but that's only practical, not necessary.

However, removing the naturalist presupposition, the proof comes to an end pretty quickly when you look up the definition of intelligence.

Sure. If you carefully (or accidentally) construct your definitions to exclude non-human activities - as we do with intellectual property, for example - then no AI/LLM will ever qualify. Essentially, a very similar argument banned women from getting any higher education in most parts of the world just a century or two ago, and still does today in many places. See also: self-fulfilling prophecy.

 

[ngtwolf]

Once the AI fad is over those nuclear plants can power something useful.

I'm still waiting for the internet fad to be over, yet here we are.

Why the Web Won't Be Nirvana

The truth in no online database will replace your daily newspaper, no CD-ROM can take the place of a competent teacher and no computer network will change the way government works.

www.newsweek.com

Seriously though, AI is just a tool, and it's not going away. The tech is definitely still in early days, which is why we can find so many flaws with it... However, as a parallel, we can look at a failed dotcom business like webvan which was a terrible idea in the late 90s, but is a core business of many online grocers (or amazon) now... it just took time for both the internet and other innovations to ramp up to make these ideas possible.

 

[cjs]

Seriously though, AI is just a tool, and it's not going away. The tech is definitely still in early days....

If you feel that, seventy years on, it's still "early days," it sounds to me that you're saying it's quite likely we won't see AI success in our lifetimes.

If you're referring to just this round of AI enthusiasm, I'd like to hear an explanation of why this is different from the last four major rounds of it, all of which massively failed to live up to the claims people were making about them.

Religion is far from formally sound, yet a major part of human life.

Ah, well, there we go. Because religion is far from formally sound, we will never be able to, e.g., send a human to the moon.

But wait, I think we actually did do that (though a substantial number of people disagree.) Somehow the existence of religion was irrelevant to whether we could use logic, science and engineering to achieve this complex task.

Current-generation LLMs don't have the ability, that is true. But why do you conclude that they will never have it?

Because of their very design. Directed randomised word generation is a vastly different thing from performing logical analysis.

As a side-note: Basic image generators or autocompletion engines do not need the ability to proof logic theorems to be useful in the first place.

No. But I posit that if you don't have the ability to do simple logical analysis, you don't have intelligence.

LLMs execute on Turing machines, operating with a well-defined set of rules, consuming a well-defined set of inputs and produce a well-defined set of outputs. That makes them computer programs, by definition. If you can train a computer to follow rules, you can also train an LLM running on that computer to follow rules.

No. If you're a programmer you can replace the LLM with another program running on that machine that can do things, such as logical analysis, that the LLM software is incapable of, but then you're no longer running an LLM. You can't train an LLM to do logical analysis any more than you can use Wordstar 3.0 to brighten a JPEG image.

Given that LLMs are deterministic....

LLMs are not deterministic: that's a key part of their design.

Technically you could make an LLM deterministic through some careful work, by both ensuring all the randomness it uses in text generation comes from a known seed that you set (no use of PRNGs seeded from /dev/urandom!) and you use a fixed set of training materials rather than downloading random stuff from the Internet, but even there, the massive level of permutation that's going on makes it effectively non-deterministic anyway from any practical point of view. It's not going to change the situation that nobody can tell you why an LLM produces a particular answer. (This has been a problem dogging machine learning from the start.)

 

[ngtwolf]

If you feel that, seventy years on, it's still "early days," it sounds to me that you're saying it's quite likely we won't see AI success in our lifetimes.

If you're referring to just this round of AI enthusiasm, I'd like to hear an explanation of why this is different from the last four major rounds of it, all of which massively failed to live up to the claims people were making about them.

Well, AI is a terrible term for what exists now. It's just the term they use, but no, i'm specifically referring to LLM's like ChatGPT/Copilot/etc. I definitely see how they are great at summarizing documents, searching documents and creating first drafts of documents. I basically think that the way the current LLM models work will replace search as we know it and will help with gathering and organizing the vast amounts of information out there. Anyway, I believe that use is here to stay and it will only get better. The biggest issues with them right now isn't even the hallucinations or any of that, but rather the limited context windows that allow much more in-depth refining of requests.

I also find the image generation tools very handy and those will get better as well. I've had many times that I'm designing a presentation or document where I needed an image of something and can just type in the type of image i need and it will generate it on the fly. If the image isn't quite right, i can have it alter or adjust the images until i get something more to my liking. Of course, it can definitely go off the rails and still can't seem to do text quite right, but whatever, it saves a ton of time trying to search for the right images that are the right size and trying to determine copyrights, etc.

 

[cjs]

Well, AI is a terrible term for what exists now.

I fully agree. I wouldn't call it "AI" at all.

I definitely see how they are great at summarizing documents, searching documents and creating first drafts of documents. I basically think that the way the current LLM models work will replace search as we know it and will help with gathering and organizing the vast amounts of information out there.

I have found LLMs useful from time to time, but not in any way like the hype makes it out to be. For getting information about things you don't know much about, it's far worse than Wikipedia or StackExchange. What it's mainly been useful for is providing broken examples of things that I can fix, and similar things that rely heavily on my expertise.

I think that LLMs have the potential to take over a lot of what search engines do now, but as they're currently designed, they're nowhere near as good or useful as a search engine. (Not even now, when Google has turned to crap, much less years ago when it was pretty good.) The two major problems the current LLMs have are the inability to give links to sources (though Copilot is starting to make progress on this) and that if they don't have enough information about something, they just make it up (and, almost worse, present it as if they do have that information).

Over the course of many queries over the last year or two, I've only once gotten a better result than a search engine, when ChatGPT somehow managed to find a Python Setuptools config file option I needed that I just could not find at all in the documentation. Of course it couldn't tell me where it found it; it's entirely possible that it just "hallucinated" it but it did happen to be an undocumented option. (Though I guess we should still count that as a success.)

I would love to see the apparent conversational abilities of LLMs in voice assistants like Alexa and Siri, and it seems like an immensely obvious use of the technology. (I'd love to tell Alexa, "Please use a 24-hour clock from now on" and have it actually do that.) But several years down the road, there's no sign of that happening. I have no idea why not, but I suspect that there's something going on in LLMs that makes that really difficult to do, or why would they not have done it yet? (Hell, Bard/Gemini and Google Assistant are in the same company, and they can't seem to put the two together.)

Anyway, I believe that use is here to stay and it will only get better.

We'll see. The business behind the LLMs is on very shaky foundations, with severely (and deliberately) overhyped expectations and no viable business model that can come anywhere near justifying the amount of money they LLM shops are raising and spending. Sam Altman, in particular, makes Elizabeth Holmes look like a dilettante. He's managed to fail upward for years, having no real skills beyond convincing people that he is good at running companies and making money. (He has proven himself good at neither.) Though with him determining the direction of development, it does perhaps explain what ChatGPT is such an amazing bullshitter.