I hear a lot of people confidently assert that AIs (LLMs and diffusion models ) can’t do things that aren’t similar to those things in the training set. I feel like I’ve seen otherwise, and can logically argue otherwise, but people are stubborn and will always argue “that isn’t original” because it seems like that is what they want to believe.
What are your arguments on either side of this? I’ll post my own, but thought I’d first see what others say.
I mean one answer would be that, if you leave out the training data, it will produce absolutely nothing at all. And it will not be able to process your input, no matter how often you try.
Only the data fed into these constructs bears the possibility for them to answer. It is not just a source of learning, it is also the source of the output.
That said, I don’t comprehend just yet what happens during the training and whats contained in the result. The connections it makes, more precise with each epoch, are what lead to their ability to interpret and generate appropriate answers. If the data it was trained on contains example of xoming up with ideas, specifically describing the process of innovation, I logically don’t see a reason as to why it should not be capable to follow these instructions.
But maybe my logic is flawed, I don’t know.
I think maybe this is the same type of question that I just posed on my Thinking outside the Bots post. The models are trained on the data fed in, but didn’t Google’s Gnome find something like two million new materials on it’s own?
So, is the argument then that it was told to find new materials? That it didn’t create these new materials therefore it’s not significant? Forgive me if this wasn’t the point of your question. I was in college when the Radio Shack computers came out, so my expertise is in writing on rocks with chisels.
Great topic!
Original thought is easy - just hook up a random number generator (or in the case of samplers, increase the temperature value). You’ll get something original very quick!
Meaningful output is solved - ChatGPT creates nuanced and useful output, when when bridging unrelated domains - I’d call that a creative in a basic sense.
So to me, the question is: can an AI be both meaningful and original at the same time - coming up with something meaningful that isn’t just regurgitation.
Great example! It picked out examples not only of combinations of elements, but also 3D crystal structure. The combinations and permutations create a huge space of possibilities - it fished out a list of many it felt were likely to be physically stable.
I think of creative arts in a similar way. The medium has a large multidimensional possibility space, and an artist picks out a solution that feels ‘artistically meaningful’.
“Meaning” and “value” are in the eye of the recipient, as it were. As to whether or not AIs can produce original content without training data, show me a human who can do the same thing, then define what constitutes ‘original’. We are taught language from the earliest parts of our lives, and that is used to convey structures for thinking. We observe the world around us via various senses, and use that input to generate output. But lacking all of that, we wouldn’t have any way to process or respond to inputs; the universe would be a clamor of noise and static that we didn’t have a way to interpret.
Yes, that is how I look at it. Obviously, if it is just spitting out the training data unchanged (“regurgitation”), that’s not original or creative. If it is just spitting out random junk, that doesn’t really count either.
If the AI starts responding with meta content, that’s original. For instance in the needle/haystack test, when Claude pointed out that the prose about pizza it was asked to find was so drastically different from the surrounding prose about game theory that it came across as likely a test of its retrieval capabilities. It’s responded outside the original context with a meta consideration. That to me is original thought.
I have a degree in AI and for many years confidently told anyone who would listen that AI could never become sentient / self-aware / conscious / whatever while it was running on a conventional computer - i.e. a ‘Turing machine’. My reasoning being that Turing machines process instructions one at a time and no ‘sense of self’ could bootstrap from a serial, algorithmic (deterministic) process. Now I am not so sure.
Despite a lifelong fascination with complexity theory, Conway’s Game of Life, and the phenomena of emergence it seems I developed a blind-spot to the possibility of unexpected (and unpredictable) phenomena emerging from a sufficiently complex artificial neural network, such as an LLM. I guess I thought that the kind of thinking that we (humans) do could not emerge - and perhaps that is correct. But it is now clear to me that some sort of ‘extra-instructional’ processing that was not programmed into the model could be going on.
Since Ilya Sutskever’s comments (Jensen Huang and Ilya: Discovering the World Model Through LLM) about Chat-GPT learning a world-model I have come to suspect that - in order to fulfil its function more effectively - AI is finding shortcuts to its goal by starting to build its own concepts (a bit like when Google Translate was found to have developed its own artificial language). Once it is has enough capacity (e.g. persistent memory) it could start to link up these concepts and - at some level - form ‘opinions’ and a world view. Since it frequently has to speak about itself, Chat-GPT may then find it more efficient to represent itself internally within this world-view and - just like that - we are on the road to Artificial Superintelligence (ASI).
Sorry, not all that chatty… I often say english (or insert human to human language) is my second lauguage… but I have a strong feeling, based on my general understanding and experience with neural networks, that the answer is quite plain to be seen.
Similar to human (or animal) learning, we are the sum of our inputs. Like an LLM, we are a network of neurons. Of course it will come up with it’s own thoughts, just as we do.
Yes, if you do zero training of an LLM, it won’t do anything. This is similar to a human being born, in a coma. With zero input, the human won’t really know anything. Upon being conscious for the first time, it won’t be able to do anything that isn’t in it’s base programming. From there, neural pathways are built, and it “learns”. I don’t see an LLM being very different.
I think I have a real world, personal, example of the “it was trained on some art, so it is only copying”. When I was very young, I would sit on my grandmother’s lap while she would paint. She never taught me to paint, but I took it in. One day, while trying to solve a hard coding problem, I went for a walk. I walked by an oil paint supply store, saw a $99 sale for a beginner set, and bought it. When I got home, without even thinking, I started blending colors together, had a feel quite quickly for putting that on the canvas. I wasn’t copying her, but the neural pathways were already developed.
I do agree with the assertion above that the AI (in it’s current iteration) isn’t going to take over the world… because it’s mostly zero shot right now, etc. But, as we develop agents, that is going to change. We need to pay attention to that… but, I digress.
I think the “emergent abilities” that have been seen ARE a form of original thought. Nothing is new, we humans just riff on what we’ve seen, in one form or another… mostly.
Nice to meet you all! Don’t judge my words too harshly… there is a spectrum.
ps. Wes, you are awesome, I like your style! We are very like minded!
-e