I wouldn't feel that smug considering these are single prompt generations on a pretty small project.
As Two Minute Paper's always says, it's not just about what this looks like at the moment, it's about what this might look like another three breakthroughs down the line.
While you can't guarantee further breakthroughs, at the rate of advancement and pace of improvement, you would have to be brave to bet on no further breakthroughs.
I agree with you, but I think it skips over the fundamental point of the demo - that this is possible at all. The door is unlocked. I expect where commercial interests will take this over the next year or two, even without further "model breakthroughs" will be enough to change how many devs engage in game development.
> Basically it's because what "AI" can do is extremely different from what "AI evangelists" claim it can do.
You always have people at both sides of the aisle though - people who say it can do much more than it can, and people who say it can do much less.
It's the same with all technologies - robotics, crypto, drug discovery, the internet, digital cameras, quantum computing, 3D Television, self-driving cars - it was probably the same with the steam engine. All of these will have had people who said that the technology would be useless and die (e.g. Napoleon and the steam engine), and others that would have said it was totally transformative.
Pointing to people who hold extreme opinions 'for' a particular technology that are overly-bullish, and then dismissing the technology based on that, isn't a particularly good strategy in my opinion.
> Am I naïve in expecting Artificial Intelligence to be smart? Is my interpretation of the word “intelligence” too literal? And when an AI behaves stupidly, who’s to blame? The programmers or the AI entity itself? Is it even proper to make a distinction between the two? Or does the AI work in so mysterious a way that the programmers need no longer take responsibility?
IMO this is a programming/prompting failure - not a failure in the general capability of 'AI'.
We can prove that an AI can understand this with a basic prompt:
This is a minimal base prompt, with no fine-tuning, with the same user prompt, which shows that an AI will respond correctly by default. Presumably either the AI they are using is a weak model, or their prompt is encouraging the model against this (e.g. maybe the prompt says 'return one song based on the suggestion, and then songs from similar artists after')
> I’ve heard people claim that an AI can compose music. But how can that be when it can’t even grasp basic concepts in music?
Trying to infer the underlying capability of AI to generate music based on a badly-prompted Spotify DJ feature is always going to have it's limits. The proof of 'can AI compose music' will be in the eating of the pudding. AI models have already been able to compose classical music to some extent, and can grasp music theory, so after this point it's just going to be a matter of quality/taste.
Your reference to prompting is pretty disgusting since you try to shift the blame to the user. All the prompts were crystal clear. Trying to shift any blame on user error is non-sensical stupidity or dumb manipulation in this case.
Also, might I recommend looking at the way the world is, not the way the world might be. This is one of the ugly AI tendrils this disgusting industry is putting into everything, bringing ruin to the world. This is the actual reality of it, making the world a dumber, less interesting more stupid place.
> Your reference to prompting is pretty disgusting since you try to shift the blame to the user.
I'm shifting 'blame' to Spotify, rather than the user or the AI model - although blame is probably a pretty strong word anyway for what is probably just supposed to be a fun DJ feature.
> All the prompts were crystal clear.
We don't know what the prompt is, because the FULL prompt will be a combination of the base prompt plus the user prompt. It's trivial to show that a modern model with a minimal base prompt will return correctly (as per my original post), so IMO there is probably something in the base prompt which is encouraging the model to return differently.
I wanted to clarify the first two points, but i'll not respond to the rest of your comment as it's a bit overly-emotive (calling what I say disgusting, rambling about the downfall of society as a whole etc).
> it was a skill issue on the part of the Spotify engineers writing the internal system prompt for their slop DJ
Spotify are currently making a big deal about not writing any code - I attended a webinar this week where one of the slides proudly trumpeted this fact:
“
0 lines of code
Spotify's best engineers have not written a line of code since December.”
A full decade since spotify wrapped was released and we still have the grand unified … menu with a seemingly randomized list of actions that takes hundreds of milliseconds to load depending on network conditions. And buggy jams that desync constantly. And it’s way too easy to accidentally clear the entire queue. I could go on, wonder what the hundreds of PMs are doing?
Bunch of clowns coasting on their moat instead of building an actually good product.
> Your reference to prompting is pretty disgusting since you try to shift the blame to the user
Users are often to blame in many varied cases and there should be no taboo around discussing this. I think maybe some people hear that you should never blame rape victims for rape and then go running wild trying to apply that as a general principle of never blaming anybody who is in any way a victim of anything, even when the "victimhood" is simply some piece of trivial software not working well. But we're not talking about rape so your intense rejection ("disgusting") is completely off the mark.
its quite common for schools to issue windows laptops to staff (who use MS 365) and chromebooks to students (who use Google Classroom). The windows laptops also have no problem with google classroom of course.
I think we can safely assume that OP was picking a bit of a ridiculous hypothetical example to make a point that it’s possible for something to be deadly and transmissible, although in nature Baculovirus in Caterpillars has a similar mechanism (encourages their host to eat a lot, then climb to the top of a plant so when it turns to ooze it infects others) or cordyceps although both of these aren’t as highly transmissible as they hypothetical explode virus.
But the Black Death mixed high contagion and high mortality as an actual example that shows they aren’t mutually exclusive.
What? That's your second strawman in two comments.
Nobody said you claimed they were harmless. People are taking issue with your assertion that biological agents can be either contagious or lethal (not both), and therefore you discount its risk. This implied tradeoff between contagiousness and lethality simply is not enforced by anything in nature.
The natural emergence of a pathogen that's both highly contagious and highly lethal would be a much rarer event than the natural emergence of one that's either contagious or lethal, but we're talking about engineered pathogens. There is no reason to think that pathogens cannot be deliberately created that are both of those things.
Because humans always say 'bread' if you ask them what you put in a toaster.
And humans will always deduce that you should switch doors if you are in a hypothetical gameshow and they show you a horse behind one of the doors.
(All I mean is - an example of an LLM answering illogically is not proof that LLMs can't really think logically, as you can equally find examples of humans answering illogically and also find examples of novel questions that LLMs get right)
Humans aren't immune to getting questions like this wrong either, so I don't think it changes much in terms of the ability of AI to replace jobs.
I've seen senior software engineers get tricked with the 'if YES spells yes, what does EYES spell?', or 'Say silk three times, what do cows drink?', or 'What do you put in a toaster?'.
Even if not a trick - lots of people get the 'bat and a ball cost £1.10 in total. The bat costs £1 more than the ball. How much does the ball cost?' question wrong, or '5 machines take 5 minutes to make 5 widgets. How long do 100 machines take to make 100 widgets?' etc. There are obviously more complex variants of all these that have even lower success rates for humans.
In addition, being PHD-Level in maths as a human doesn't make you immune to the 'toaster/toast' question (assuming you haven't heard it before).
So if we assume humans are generally intelligent and can be a senior software engineer, getting this sort of question confidently wrong isn't incompatible with being a competent senior software engineer.
humans without credentials are bad at basic algebra in a word problem, ergo the large language model must be substantially equivalent to a human without a credential
thanks but no thanks
i am often glad my field of endeavour does not require special professional credentials but the advent of "vibe coding" and, just, generally, unethical behavior industry-wide, makes me wonder whether it wouldn't be better to have professional education and licensing
And that many mathematicians got monty-hall wrong, despite it being intuitive for many kids.
And being at the top of your field (regardless of the PHD) does not make you immune to falling for YES / EYES.
> humans without credentials are bad at basic algebra in a word problem, ergo the large language model must be substantially equivalent to a human without a credential
I'm not saying this - i'm saying the claim that 'AI's get this question wrong ergo they cannot be a senior software engineer' is wrong when senior software engineers will get analogous questions wrong. If you apply the same bar to software engineers, you get 'senior software engineers get this question wrong so they can't be senior software engineers' which is obviously wrong.
I suspect they aren’t doing this (just) to avoid fees - it’s more about national security in a world where the US might stop being a reliable ally, and in a world where the US has used withdrawing Visa / Mastercard as a strategy to weaken enemy economies.
I think the recent stories of the International Criminal Court judge being forbidden to use VISA and Mastercard, making his life somewhat more challenging, did make some politicians aware of the risks.
The rest of the world isn't free riding - the USA has just setup a market where there is very little bargaining power for consumers because of how the US medical market and insurance works.
Novo and Eli are still making plenty of money in Europe where these drugs cost a fraction of the price, and where there aren't other significant suppliers for GLP-1's like is being implied.
No, they're free-riding. If drug companies can't charge higher prices in the US, they will do less drug development. Everyone involved in the business/investing side of pharma knows this; it's not even an argument.
I think we have a different definition of free riding.
If you and me both buy the same car, but i'm better at negotiating than you and get a lower price, I'm not free riding because you 'funded the design of the car with the extra money you paid', you are just bad at negotiation.
In this metaphor, the car manufacturer only invested R&D in new models because it expected to be able to recoup that R&D spending from my higher purchase price. If I start paying the same low price for cars as you do, the manufacturer stops investing as much in R&D for new models. Your access to new models was free riding on my higher prices.
When we all negotiate lower prices, we get fewer new drugs. Maybe that's better than the status quo (for Americans). Maybe it isn't.
As Two Minute Paper's always says, it's not just about what this looks like at the moment, it's about what this might look like another three breakthroughs down the line.
While you can't guarantee further breakthroughs, at the rate of advancement and pace of improvement, you would have to be brave to bet on no further breakthroughs.
reply