In addition to remote monitoring, central control over large systems clearly has benefits: quickly and automatically spinning up a power generation in one location in response to an outage or just increased demand in another one, or conversely spooling down generation in response to a large demand spike going away (many factories need to notify the grid before starting up or shutting down), shutting down water/gas flow upstream of a detected leak, or yes, working from home, which lots of people here regularly argue is a good thing.
You could absolutely make the case that it isn't worth the risk, but that isn't the same as not having benefits.
No, and it's not strictly impossible to correctly build out a SOC / SIEM and ingest all the logs you want and pay for the right engineers to make sure it all works correctly. But damned if anyone manages to do a great job in this area. It's too complex and too expensive, so almost everyone settles for "best effort."
A lot of problems are easy conceptually, but we can't manage to tackle them.
I took humans thousands of years, then hundreds of years, to come to terms with very basic concepts about numbers.
Its amazing to me when people talk about recombining things, or following up on things as somehow lesser work.
People can't separate the perspective they were given when they learned the concepts, that those who developed the concepts didn't have because they didn't exist.
Simple things are hard, or everything simple would have been done hundreds of years ago, and that is certainly not the case. Seeing something others have not noticed is very hard, when we don't have the concepts that the "invisible" things right in front of us will teach us.
It's why the invention of teaching has been so important. Took a long time for humans to develop calculus. A long time to then refine it and make it much more useful. But then in a year or two an average person can learn what took hundreds of years to invent. It's crazy to equate these tasks as being the same. Even incremental innovation is difficult. You have to see something billions of people haven't. But there's also paradigm shifts and well... if you're not considered crazy at first then did you really shift a paradigm?
And yet it is still taught in less than optimal form, lacking algebraic closure in ways that are completely unnecessary.
It isn't a secret, but the percentage of people who don't know that, plus the percentage of mathematicians who vaguely or more directly know that, but habitually use the broken, more difficult (i.e. less algebraic) notation is ... virtually everyone.
I am not trying to pick on calculus, this is everywhere. Important and useful concepts are right in front of all of us, that we don't see even in the context of what we are relatively fluent with.
Because we learn quickly, where we have (almost always inherited) the right preparatory perspectives (earned over lifetimes by others), we vastly overrate our ability to reason independently.
Were I to guess they're talking about the different derivatives. Here's at least something that might introduce you to some of the shortcuts people take but it's far from complete [0] (you can probably find more if you search things like how physicists use the derivative wrong. (I make this critique as someone with a degree in physics too))
I often say that math is taught through a game of telephone. It's a fanatic example of the problem with "I just care that it works" type of attitudes. The problem is if that's your actual belief then you wouldn't be saying that because you'd need to dig deeper. Caring about it working is exactly the reason people do did deeper and bring up issues. The reason things fall apart less in math is because the language was specifically invented to make miscommunication difficult. That's why it's overly pedantic. That's why we use formal languages rather than natural ones. So we should rephrase "I just care that it works" is that it's actually "I just care that it works for this exact case." It makes it easier to see the problem. If you don't know the subject in more detail then you can't actually know if it breaks in that use case. The broken parts are completely invisible to you! Which undermines your own stated goal.
This goes for a lot more than math. But being a formal language it's just easier to point things out and how people misunderstand. If you're an expert in any field you've probably see this same phenomena in that domain though. People having over confidence and their refusal to get deeper knowledge actually just undermines their whole goal. I'd honestly call this a form of Murray-Gell-man Amnesia
OpenAI themselves must not have a "reasonable definition of L", then. Their own papers and press releases refer to GPT-2 (from 2019) as a "large language model".
Yes, and 1.5 billion parameters meets no reasonable current definition of large. It would be considered a tiny language model. OpenAI themselves refer to their small/fast models as small models all over their documentation.
The term doesn't change its meaning because something new comes along.
The point of the term "large" is to highlight the massive parameter count (compared to traditional statistical models, where having 1.5 billion parameters was basically unheard of). It leads to the "double decent" phenomenon that allows them to generalize in ways traditional statistical models can't.
The idea that the "large" descriptor was just a subjective exclamation, like "oh wow this model is pretty large ain't it", is revisionism.
yes, it does. That's why OpenAI refers to it's small models as small. They are just so different. The capabilities have changed dramatically. The use cases are wildly different. The architectures are quite different. Even the core idea of attention is different. Training them is materially different. Serving them is materially different. A 1.5 bill parameter model from 2019 is so different from today's LLMs that they really don't have much in common. What we have now is quite similar to what we had a couple years ago though.
Sure we do, since Fei-Fei Li and team created that annotated dataset, which allowed to train first LLMs. So LLMs are here for more than a decade already.
Fei Fei was annotating images... the second L in LLM is for "language". The first language models named LLM at the time were trained on language data, with an objective function of predicting the next token. It had nothing to do with the imagenet data. Imagenet data was used in... vision models.
The attention is all you need paper didn't ever use the term LLM or large language model because the phrase didn't exist in industry.
When people say this what they mean is that we've had plausibly useful LLMs for around three years, and I would say that is basically true. The stuff before 2023 could barely be classified above the level of an interesting toy.
The closer you get to releasing software, the less useful LLMs become. They tend to go into loops of 'Fixed it!' without having fixed anything.
In my opinion, attempting to hold the hand of the LLM via prompts in English for the 'last mile' to production ready code runs into the fundamental problem of ambiguity of natural languages.
From my experience, those developers that believe LLMs are good enough for production are either building systems that are not critical (e.g. 80% is correct enough), or they do not have the experience to be able to detect how LLM generated code would fail in production beyond the 'happy path'.
> The closer you get to releasing software, the less useful LLMs become.
Which is _always_ the case with these things, honestly. Remember Ruby on Rails? Make a Twitter clone in half an hour by just writing some DSL! Of course, in reality Rails was _not_ a productivity revolution, and making _real_ software which had to be operated at scale and maintained, and work properly, in it wasn't much easier than it had been previously.
The amount of "apps" I've had dumped on my team that are everything from un-releasable to deployed on some random shit-cloud we haven't approved (vercel comes up a lot). If you needed hand holding to release things or had to throw software over the fence to others to "productionise" etc then you probably don't know what you're talking about.
I would estimate that out of every 200 lines of code that Claude Code produces, I notice at least 1 issue that would cause severe problems in production.
In my opinion these discussions should include MREs (minimal reproducible examples) in the form of prompts to ground the discussion.
For example, take this prompt and put it into Claude Code, can you see the problematic ways it is handling transactions?
---
The invoicing system is being merged into the core system that uses Postgres as its database. The core system has a table for users with columns user_id, username, creation_date . The invoicing data is available in a json file with columns user_id, invoice_id, amount, description.
The data is too big to fit in memory.
Your role is to create a Python program that creates a table for the invoices in Postgres and then inserts the data from the json file. Users will be accessing the system while the invoices are being inserted.
And that's why you ask for a high level plan for something like that before you let the agent write any code. Then you review the plan for flaws, revise it, and prompt the system to fill out more details for each step. Repeat as necessary. Yes it's slow, but it's the best way of using this "glorified autocomplete" to ease and speed up real work.
If you are trying to build something well represented in the training data, you could get a usable prototype.
If you are unfamiliar with the various ways that naive code would fail in production, you could be fooled into thinking generated code is all you need.
If you try to hold the hand of the coding agents to bring code to a point where it is production ready, be prepared for a frustrating cycle of models responding with ‘Fixed it!’ while only having introduced further issues.
Any sufficiently complicated LLM generated program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of an open source project.
We had an effort recently where one much more experienced dev from our company ran Claude on our oldish codebase for one system, with the goal of transforming it into newer structure, newer libraries etc. while preserving various built in functionalities. Not the first time this guy did such a thing and he is supposed to be an expert.
I took a look at the result and its maybe half of stuff missing completely, rest is cryptic. I know that codebase by heart since I created it. From my 20+ years of experience correcting all this would take way more effort than manual rewrite from scratch by a senior. Suffice to say thats not what upper management wants to hear, llm adoption often became one of their yearly targets to be evaluated against. So we have a hammer and looking for nails to bend and crook.
Suffice to say this effort led nowhere since we have other high priority goals, for now. Smaller things here & there, why not. Bigger efforts, so far sawed-off 2-barrel shotgun loaded with buckshot right into both feet.
Not to take away from your experience but to offer a counterpoint.
I used claude code to port rust pdb parsing library to typescript.
My SumatraPDF is a large C++ app and I wanted visibility into where does the size of functions / data go, layout of classes. So I wanted to build a tool to dump info out of a PDB. But I have been diagnosed with extreme case of Rustophobiatis so I just can't touch rust code. Hence, the port to typescript.
With my assistance it did the work in an afternoon and did it well. The code worked. I ran it against large PDB from SumatraPDF and it matched the output of other tools.
In a way porting from one language to another is extreme case of refactoring and Claude did it very well.
I think that in general (your experience notwithstanding) Claude Caude is excellent at refactorings.
Here are 3 refactorings from SumatraPDF where I asked claude code to simplify code written by a human:
I hope you agree the code written by Claude is better than the code written by a human.
Granted, those are small changes but I think it generalizes into bigger changes. I have few refactorings in mind I wanted to do for a long time and maybe with Claude they will finally be feasible (they were not feasible before only because I don't have infinite amount of time to do everything I want to do).
“I want this thing, but in a different language” seems to be something that the current generation of cutting edge LLMs are pretty good at.
Translating a vibe is something the Ur-LLMS (GPT3 etc) were very good at so it’s not entirely surprising that the current state of the art is to be found in things of a “translate thing X that already exists into context Y” nature.
Was software ever a moat? Software typically only gave companies a small window of opportunity to turn a fleeting software advantage into a more resilient moat (network effects, switching costs etc.)
Yes, I would argue good (stable, fast, easy to use) software was somewhat of a moat and much harder before coding agents.
Stripe, Square, Shopify, Google, all thrived in some part because their services take a hard problem and make it easier to use. Now more people can take a hard problem and make it easier to use.
All you have to do is look around (esp 5+ years ago) and see the many many BAD, unstable, hard to use, slow, etc versions of these companies
Windows' moat was not the operating system code, but that they were able to get distribution via IBM, and then grow an ecosystem of applications that were targeted at Windows, which created a snowball effect for further applications.
In what way is the long term impact of LLMs being underestimated? If anything, it seems that it has been overestimated in the past years and that something other than LLMs will be needed to reach the original scaled LLM hope of AGI.
Back when the Internet was America online and some CGI bin perl scripts, there were a lot of very lofty things said about the potential of the Internet in the future. I don’t remember any of them predicting the power of the tech would have over business, politics, media, and hours of every single day for billions of people. Even without AGI, it’s quite possible that were still underestimating. The effects of predictive, probabilistic computing 20 or 50 years from now.
The internet alone didnt change sh!t. Without smartphones, unified app stores, cellular network innovation et al internet traffic would not be so high.
Funny how people leave this stuff out. Yawn. Basic simpleton analysis and takes.
The Internet created the backbone that allowed for rapid experimentation in communications technologies, and created the ability for anyone to create and share technologies and reach a huge audience very quickly.
Without the Internet, most consumer electronics would have been far more expensive to build, and would have been strictly controlled walled gardens, but the Internet in general and the Web in particular allowed so many inventors to flourish. Ever since that Genie was let out of the bottle, corporate and government interests have been trying to put it back in, and most companies are trying to build and reinforce walled gardens under the banner of unified app stores that extract insane rents.
Unfortunately, it seems investors now think that all paid software will be replaced by AI generated software, somehow open source projects laundered through generative AI models should finally convince enterprise customers to go with free.
1. Click fraud is rampant, and the ad platforms seem to not care in the least as long as they can drain your budget.
2. The ad bidding process is completely opaque, and no auditing seems to be done, my inclination is that there is large scale fraud here too.
reply