> If you believe it is dangerous, you should be dedicating yourself to STOPPING others from making it
I don't think anyone has been more successful in promulgating AI safety
There are groups like MIRI who tried what you're sugesting, where they make no AI and just push for AI regs, and they have been relatively much less successful
Can one really not imagine a case where the cheating machine being used by students is a bad thing for teachers? Does everything have to be "politically motivated"?
There are tons of reasons AI is actively making the school system worse (amongst many other aspects of society). Immediately jumping to "political coalition thing" seems strange.
Unions are always against whatever management wants. Then it becomes a bargaining chip for what the union wants. That's how collective bargaining works.
I think unions in industries where their workers are at risk of eventually getting replaced by AI are pretty universally against it, because protecting the jobs of members is the whole purpose of a union. It’s like how the teamsters are against self-driving cars.
I guess I should qualify that, many professors are ambivalent regarding AI, but some view it as an existential threat to their profession. Most here are scrambling to figure out ways to work around it if not incorporate some AI into the curriculum, since every student has access to ChapGPT and many also have access to CoPilot.
Why wouldn't the political coalition of teachers not be a "real" teacher-related reason? It is not illegal, at this point in time, for teachers to oppose AI for political reasons.
As others note, there are a lot of reasons for teachers to refuse or hate AI, though in my experience most don't know shit about it and just want students to stop using it as an expedient. I, for instance, take a look at the tiny Dell cubes that have barely powered our Windows workstations and hilariously bedraggled Prometheus units and anticipate "well, we can't even afford to update these pieces of shit, so I suppose as a 'Microsoft shop' we'll be on a upgrade path to CoPilot-enabled cloud computing or some bullshit like that, then it'll really be all over" so my primary concerns are infrastructural. But god yeah the AI writing I get, jesus. These kids think they're driving around in the AI equivalent of Lambos, but free tier CoPilot is a used 2017 Chevy Cruze.
You don't need to make this judgement; it's fixed by the colorspace you're working in.
First, figure out what colorspace the processing needs to happen in. Usually this is linear RGB.
Then, figure out what OETF and EOTF your input/output format use. This will be something like PQ or HLG. This will exactly specify the meaning of each integer value.
This fixes the choice of representation and conversion.
> there will be hardware capable of running frontier models
The current frontier? Sure. The frontier then? No - obviously that frontier is going to keep consuming available datacenter compute capacity, which will be better
Author here. I have never said that phrase before this blog post and certainly understand the absurdity of it. I certainly don't mean that you need something biological or whatever consciousness might or might not be.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.
For a lot (most) of what we do with programming, the process actually doesn't matter. I understand you are a real ass dude who is in this shit for the love of the game. I respect that. You are a true artisan and exist in a kind of rarified space. There will always be a place for people like you and in some senses you are correct - you are not replaceable by any AI as they currently function today.
However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
LLMs are even more useful to experts who know the limitations. However the process matters even more if you want to build robust and scalable secure systems that generate millions of dollars and can explain that accurately to high value clients.
I do not want a $10M - $100M dollar issue (lawsuits) because I admitted that I don't understand why a breach happened after using a coding agent. Responsiblity and reputation can't be vibe-coded.
So:
> However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
There's a vast difference between code that works as a prototype vs how it works in production. I don't think you would trust anyone with no experience to fly a commercial plane with them vibe-coding a flight simulator without knowing the process of becoming a pilot.
It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done.
Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern.
The LLM in this use case is perfect because almost everything involved is text based, and the model is able to take in all the expressive that is language.
> Is this any different than how a PM gets a programmer to work on a project?
Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.
I think they have the capabilities to execute a well defined plan. If you truly don't believe that, then you I suspect any work you do as a programmer will not survive the coming changes.
It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall.
As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.
I would suggest you examine current harness memory persistence. Any reprimand you give your model will be remembered, in the same way a puppy that has a bad social experience will become more shy.
They will not save you from every pitfall, but that isn't the point; engineers walk into pitfalls all the time. This can get you in, and out, much much quicker.
But its not useful because even humans are like that - a bunch of neurons slapped together. Overall a tired analogy that is more suited to stay in 2024 where it belongs. Right now it is clear that it is _much_ more than a statistical model semantically. It is misleading to claim it is _just_ that just like a human is _just_ a statistical model.
That's a very good way to describe what they do (better than 'AI') but ironically, it really well explains the mechanism, and how they are in fact able to 'code so well' which is contrary to the authors own premise.
Agents code extremely well.
They're not particularly good at 'architecture' and I think that's where his specific concerns about 'not being able to see the problems' arise - the issues are are almost never in the syntax, because the AI writes perfect code. The issue is that it's not doing exactly what you intended.
Instead of 'missing the target' ... it's 'hit the wrong target perfectly'.
Any senior developer working with AI daily should be able to have a baseline intuition for all of this, and would therefore reject the hyperbole of the premise 'it can't code!'.
Of course it's producing gargantuan amounts of slop - that's not because 'it can't code', that's something else entirely.
> Of course it's producing gargantuan amounts of slop - that's not because 'it can't code'
That is precisely because it can't code! Or rather, it's because it can't reason, or understand things, which in turn means it can't code. The output of LLMs is sloppy because they have no understanding of what they are doing.
That people will say 'Code Me An App' and expect some kind of magical results, will be more common than not, but it's no way evidence that the AI can't code.
Given a sufficiently detailed prompt, the AI will produce almost whatever you ask it within a certain scale.
As sure as the sky is blue.
And it will make perfectly compilable code usually on the first prompt.
Obviously, it can code.
Obviously, it can 'synthetically reason' about the code.
You can point it an arbitrary code base and it will give a better overall assessment than most humans.
Is it fallible? Obviously. Is it limited in scope? Obviously.
It's the mantra of the AI skeptics. Sounds so clever because it's technically true. Just like humans are just piles of oxygen, hydrogen, nitrogen and carbon atoms, along with maybe a dozen or so other trace elements, none of which have any intelligence or will or desire to do anything - hence humans cannot possibly be intelligent or have any free will.
You're already writing Rust in a very different style if you're writing the type of code that gracefully handles allocation failure. It's to Rust's immense credit that this type of coding is actually fairly well-supported (unlike in Go), but you're already a bit off the beaten path for stuff like error handling.
Every time I've ever seen a tech worker's union, it's always some sort of political experiment rather than legitimately advocating for the interests of the workers it nominally aims to represent. E.g. the Google AWU-CWA union just did a bunch of political stunt stuff, no salary negotiation or anything useful to the modal Google worker.
Partly because they couldn't because they didn't organize in a way that let them because... well... one could speculate.
> Alphabet Workers Union (AWU), also informally referred to as the Google Union, is an American trade union of workers employed at Alphabet Inc., Google's parent company, with a membership of over 800, in a company with 130,000 employees, not including temps, contractors, and vendors in the United States. It was announced on January 4, 2021, with an initial membership of over 400, after over a year of secret organizing, and the union includes all types of workers at Alphabet, including full-time, temporary, vendors and contractors of all job types.
It's trying to cover too many different groups with competing interests (FTE, temp, vendor, and contractor).
Hypothetical negotiations that would favor FTEs may disfavor vendor or contractor contracts. That inherent conflict of interest in the negotiations would mean they can't negotiate for any of them on those matters. Also, the less than 1% of the people belonging to the union would mean the union can't represent them in collective salary negotiations either.
Of the represented group (say if they only organized for FTE tech workers), they would then have needed 50% + 1 of the employees in that classification to vote to have a union. It is possible - https://kickstarterunited.org for example (and yes, they are having trouble - but they are negotiating on working conditions and pay).
---
Various "we should have a union" strings typically have been people wishing for one that is cross industry that they don't have to do anything. While industry wide union organization can exist (Kickstarter United is OPEIU - https://www.opeiu.org ) it is the local part that people forget... Kickstarter United is OPEIU local 153.
If people want a union, they need to organize at their company and get that 50% + 1 vote there.
When it is easier to switch jobs than it is to spend the several years to organize and negotiate a contract, the power of a union is diminished.
Is it more moral to allocate goods to the people who are willing to pay the most for them, or to the people who have good scalping bots/happen to get lucky
I don't think anyone has been more successful in promulgating AI safety
There are groups like MIRI who tried what you're sugesting, where they make no AI and just push for AI regs, and they have been relatively much less successful
reply