There are endless posts on HN about how Macs are good for LLMs. Somoene is running some model on their Mac for local inference.
What the posts dont mention is how unusable that experience is with dogshit slow tokens/second. To make a local model usefull you need to run the highest parameter models at 100+ tok/sec, otherwise you are just better off paying for cloud inference. So either all those people are dumb as hell, or Apple is doing clever advertising. And I personally have more faith in the tech sector.
Exhibit A: Here we have someone who's been sold war and inequality as the drivers of progress. Coincidentally, their obedience was deemed fiscally advantageous in order to advance the interests of the members of the 1% club.
I think that mainly changes the nature of the threat rather than anything else.
Even if we didn't already have any wealth inequality or consolidated power, they're already starting to become capable of creating those things for the first people to think to ask for it. Even in little ways, like telling you how to make a laser microphone and then doing voice-to-text and sentiment analysis on all the voices you hear.
Remember, we had social networks before Zuckerberg monetised it and put ads in between every third message you saw from your friends; and not every third that they wrote, every third that Zuckerberg's ad system deigned to show you to keep you hooked.
We had machine learning systems even back then. It was still AI, it just wasn't able to evaluate or respond to freeform text.
> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims.
What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.
If someone told you that your house is on fire, would you just stay there asking how you should vote because there is no call to action? Someone with, as far as it looks, good knowledge is giving you his insight. Use that information as good as you can and act responsible. No one has the responsability to tell you what to do.
As misguided and misinformed as some of it may be, the "anti AI" and "no more datacenters" pushback in the US is actually working to some extent. Multiple states and locales have issued construction/permit moratoriums already. If you think it's a threat, help slow it down.
Apply your analogy all the way through though. When your house is on fire, the last thing that should happen is the firefighter shouting at you that you’re doomed and based on their expertise on fire it’s definitely gonna kill you. But this analogy is even more hilarious when you consider that sensible people have fire distinguishers, fire retardants built in and all sort of measures for this scenario. The more think this through, right?
> No one has the responsability to tell you what to do.
Oh they absolutely do! If you’re making what you claim to be a god/AGI/whatever using the result of our (humanity’s) work, then you surly are responsible for the outcome and everything that happens in between. I find, just rolling over, to not be a good example I’d like to follow.
Minimum, we should do everything ready to make Plan A of AI 2040 possible. Start by reading it: https://ai-2040.com/
So that means things like protesting so political pressure is to not build unaligned superintelligence, setting up tech for monitoring compute, creating conversations / alliances geopolitically on this esp China/US and so on.
Read the plan and think - what does this need to happen? How can we have scenario A or S instead of scenario D?
Practically, join PauseAI, StopAI or ControlAI or any AI existential-risk or pro-alignment group you can find. There's a lot of it - ask your AI for ideas!
Why does everything have to be so black and white? It always either there's no threat at all or we all need to panic. How about be open to a reasonable discussion on potential outcomes and ways we can minimise risk?
There's some irony here because despite how many times climate change has been mentioned in this thread the current reaction mirrors climate change discourse with the majority of the thread denying the possibility of real AI risks and not even considering it as an intellectual question.
It doesn’t have to come to this. Seems far fetched. If this is a stunt (which I’m not saying it is) the reason could be that he wants to found his own AI company. If I see in a few months that happens, then I’d be more inclined to think that this was just hype.
Shouldn’t this very capable model they’ve developed be able to identify promising problems? That’s what I’d expect from how the model is being presented and advertised.
That's more or less what they did according to their announcement. They fired it at a whole bunch of high end math problems and merely concentrated all efforts on one after it made some promising progress.
It seems like that’s the opposite of what happened. They started attacking the problem when they got a wind of a possible solution from certain individuals.
But they didn't know which problem had a possible solution. So they fired it at a huge amount of problems and then merely focused on the one that turned out to be promising. The model found the promising path itself, they just re-allocated the available resources once it became apparent.
It does not align, though, with the narrative that openai is a good steward of AI. If anything, if the world/government took the AGI narrative seriously, all openai operations (except maybe serving customer inference) should immediately get shutdown and be dissected by independent investigators to find out what is going on there and how many other such breaches exist. The fact that openai continues functioning as normal and is not immediately shutdown after repeated incidents implies that the world does not really take the AGI narrative seriously.
Evidently that’s not what’s happening and I don’t think anyone seriously expects that (considering the outcome of hugging face campaign).
It’s a marketing technique as old as GTA’s early days and it’s apparently still effective in one form or the other!
What will the government do if they are worried? They’ll ask to look at the envs, logs, prompts, harness code, etc. Questions we should be asking before making assumptions about emergent breakaway behavior by colluding AGI 1.0 super agents.
The question is less about whether astra is AGI, and more about whether this direction approaches AGI and we should worry about existential risks wrt to that. If we do, openai's situation is very concerning even if astra is not AGI because it shows a lack of guardrails and any kind of measures to prevent these existential risk scenarios. Especially if we consider that this is gonna happen gradually in steps than suddenly.
If AGI existential risks were accounted for in legislation openai would have been shutdown immediately.
Actually the question is exactly that, since it's in the marketing copy. Sure there are other questions to be asked but they are not what I was talking about. Nice Motte and Bailey though.
But whether astra itself is agi or not does not change anything in this argument or discussion of whether openai is good steward or not. The issue with agents running amok was not as much the direct consequences of them to huggingface etc (it is also to some degree, but the long term risks are more consequential), but the fact that there is no actual safeguards around AI agents, which in the future could lead to very bad scenarios. You do not need to believe that astra is agi (i dont, openai wants us to believe so) to be concerned. No motte/bailey here, as I am not defending whatever openai is saying anyway.
If you want to discuss with openai instead then send them an email or sth, in public fora you discuss with people and the views they have.
And that’s the issue the article is trying to explain.
reply