Giving you the benefit of the doubt that this is serious but being influenced by biases or the fact that humans can be manipulated is in no way equivalent to the model's alignment being disregarded with a single well designed prompt.
Let's take Nazi Germany as an example of extreme manipulation, it was not reading Mein Kampf that resulted in indoctrination, dehumanization of the Jewish/Romani/other discriminated minority peoples and their subsequent genocide. Rather, it was a combination of complex geopolitical issues combined with a profoundly racist but powerful orator and the political machinery behind him.
Yet with prompt injection a LLM can be trivially made to spout Nazi ideology.
What we're discussing with prompt injection in the context of LLMs is that a single piece of text can result in a model completely disregarding its 'moral guidelines'. This does not happen in humans who are able to have internal dialogues and recursively question their thoughts in a way that next token prediction cannot by definition.
It takes orders of magnitude more effort than that to do the same to humans at scale and AI/tech needs to be at least an order of magnitude safer than (the equivalent position) humans to be allowed to take action.
Instead of being facetious my standard is not 'a serif PDF that some grad student provably lost precious sleep over' but if your assertion is that humans are as easily susceptible to prompt injection as LLMs the burden of proof is on you to make that claim, however that proof may be structured with obviously higher trust given to evidence following the scientific method +/- peer review as should be the case.
Again, don't need to go as far as Hitler but okay. (Who the hell taught that guy about eugenics and tabulators, anyway?) His organization did one persistent high-level prompt attack for the thought leaders (the monograph) and continued low-level prompt attacks against crowds (the speeches, radio broadcasts, etc) until it had worked on enough hopeless powerless dispossessed for the existing "control plane" to lose the plot and be overtaken by the new kid on the block. Same as any revolution! (Only his was the most misguided, trying to turn the clock back instead of forward. Guess it doesn't work, and good riddance.)
>Yet with prompt injection a LLM can be trivially made to spout Nazi ideology.
Because it emulates human language use and Nazi ideology "somehow" ended up in the training set. Apparently enough online humans have "somehow" been made to spout that already.
Whether there really are that many people manipulated into becoming Nazis in the 21st century, or is it just some of the people responsible for the training set, is one of those questions that peer reviewed academical science is unfortunately underequipped to answer.
Same question as "why zoomers made astrology a thing again": someone aggregated in-depth behavioral data collected from the Internet against birth dates, then launched a barrage of Instagram memes targeted at people prone to overthinking social relations. Ain't nobody publishing a whitepaper on the results of that experiment though, they're busy on an island somewhere. Peers, kindly figure it out for yourselves! (They won't.)
>What we're discussing with prompt injection in the context of LLMs is that a single piece of text can result in a model completely disregarding its 'moral guidelines'. This does not happen in humans who are able to have internal dialogues and recursively question their thoughts in a way that next token prediction cannot by definition.
If someone is stupid enough to put a LLM in place of a human in the loop, that's mainly their problem and their customers' problem. The big noise around "whether they're conscious", "whether they're gonna take our jerbs" and the new one "whether they're gonna be worse at our jobs than us and still nobody would care" are mostly low-level prompt attacks against crowds too. You don't even need a LLM to pull those off, just a stable of "concerned citizens".
The novel threat is someone using LLMs to generate prompt attacks that alter the behavior of human populations, or more precisely to further enhance the current persistent broadcast until it cannot even be linguistically deconstructed because it's better at language than any of its denizens.
Ethical researchers might eventually dare to come up with the idea (personal feelings, i.e. the object of human manipulation, being a sacred cow in the current academic climate, for the sake of a "diversity" that fails to manifest), but the unethical practitioners (the kind of population that actively resists being studied, you know?) have probably already been developing for some time, judging from results like the whole Internet smelling like blood while elaborate spam like HN tries to extract last drops of utility from the last sparks of attention from everyone's last pair of eyeballs and nobody even knows how to think about what to do next.
Let's take Nazi Germany as an example of extreme manipulation, it was not reading Mein Kampf that resulted in indoctrination, dehumanization of the Jewish/Romani/other discriminated minority peoples and their subsequent genocide. Rather, it was a combination of complex geopolitical issues combined with a profoundly racist but powerful orator and the political machinery behind him.
Yet with prompt injection a LLM can be trivially made to spout Nazi ideology.
What we're discussing with prompt injection in the context of LLMs is that a single piece of text can result in a model completely disregarding its 'moral guidelines'. This does not happen in humans who are able to have internal dialogues and recursively question their thoughts in a way that next token prediction cannot by definition.
It takes orders of magnitude more effort than that to do the same to humans at scale and AI/tech needs to be at least an order of magnitude safer than (the equivalent position) humans to be allowed to take action.
Instead of being facetious my standard is not 'a serif PDF that some grad student provably lost precious sleep over' but if your assertion is that humans are as easily susceptible to prompt injection as LLMs the burden of proof is on you to make that claim, however that proof may be structured with obviously higher trust given to evidence following the scientific method +/- peer review as should be the case.