I think there are a few different ways you can define "memory" and "memorization" here. When folks say "memorising" in the context of AI they mean "Does they AI have chunks of its training data fully/identically inside its neural network". To say "it doesn't memorize" is _not_ the same thing as saying "it has no memory". An AI also learns abstract information devoid from its textual representation. This would be an example of memory without memorization.

And you are correct; if the current lawsuits against these corporations result in a legal precedence that training on copyrighted material is not an infringement of copyright, then yes, anyone will be able to train models in that way. (Within reason; copyright/fair use is very much handled on a case-by-case basis)

My point is that bad trained models will remember chunks of the training data, maybe a theoretical perfect model will have no such problem.

I suppose you are aware of the case with ChatGPT and Dune’s Litany Against Fear memorization and the lengths OpenAI went to try and prevent the people to reproduce the issue to prove it is happening. If not google it, do you disagree that is not memorization ? That somehow the model manages to create the exact same text each time because it memorized some abstract concepts that are not in fact the exact representation fo this text ?

At no point did I claim whether they could or could not memorize. I was disagreeing with your statement "How can the AI spit out facts about a metal band if it has no memory". No one has ever said "AI's have no memory" in the way you're using it here.

Yes they _can_ and _do_ sometimes memorize. But they do not _only_ memorize which is what you implied with the statement I've quoted.

What I mean is the fact that people claim that we do not understand how LLM or the Image Gen AI work, that there is no memory. So do you support the idea that LLMs like ChatGPT does not plagiurise because they can't memorize or not support that ?

You did it again, memory != memorization! People don't claim that "there is no memory"; the argument you're trying to make is "people claim LLMs do not memorize". Memory means something completely different. Memory: the ability to remember anything at all in any way; eg "I remember his face but I can't recall his name". Memorization: remembering things _exactly_ as you saw them, usually also implying a lack of understanding; eg "he memorized all the elements of the periodic table for his test".

When people talk about AI and memory, they're not talking about the training phase or training data. When they talk about AI and memorization, they _are_ talking about the training phase and training data.

To answer your question: I _do_ think LLMs can memorize large-ish chunks of text. But: because in normal real world usage they do not output large chunks of their training data verbatim, I don't think there's a sufficient risk of plagiarism to be concerned.

Your definitions are very subjective. Seems you define a verb to memorize like not the same as store something in memory , even imperfectly.

I perfectly understand that the LLM developers do not want the LLM to store large chunk of text but the fact that the LLM can pull the text out exactly was proven , so even if we pretend the text was not "memorized" and we use some ridiculous verb like "it was quantum probabilistic ally vectorized" the issue is not fixed. If your LLM can reproduce my poem then it memorized it, AI bullshiters can use other word to cope but I do not give a shit, you need a mathematical proven training algorithm or even better good data to train your LLM to not reproduce copyrighted material. Renaming concepts is bullshit.

Don't know what your problem is, I'm literally agreeing with you and have said three times now that LLMs can memorize. Good bye.