I'm sorry, I don't mean to insult or offend anyone. I'm just recounting my observations based on my understanding of the subject - and that is really not to disparage the amazing work that's being done, but rather to highlight the scale of the problem you have to solve when you're talking about creating something similar to human intelligence. It's entirely possible I'm wrong about this, and I would love to be proven so.
Do you disagree substantively with anything I have said, or do you just think I could have phrased it better?
Thanks for your reply. I suppose a quick way to summarize my criticism is that it reads to me like you've dismissed the strengths of ML on technical grounds, while you imply you don't have any real technical experience in the field. You make a superficial comparison between the compexity of biology and ML, without providing any real insight, just saying one has lots going on and the other is matrix multiplication.
If your conclusion is that current gradient based methods probably won't scale up to AGI, you're probably right. But if you want to get involved in the discussion of why this is true, what ML actually can and can't do, etc. I would encourage you to learn more about the subject and the current research areas, and draw on that for your discussion points.
Otherwise, it comes across as "I once saw a podcast that said..." type stuff that is hard to take seriously.
No doubt I come across as condescending, please take what I say with the usual weight you'd assign to the views of a random guy on the internet :)
Actually you do have to be an expert to make sweeping statements with any credibility in a young field making advances every day. Huge ones and surprising ones every year.
If you can’t characterize the technical problem that creates a limitation then you are just expressing an uninformed opinion.
Not to get into the rest of the discussion, but I disagree with the classification of ML as a young field. AI is an established field and I would argue that nothing in modern ML is _fundamentally_ so different that it would justify classifying it as a new field.
Yeah, it's almost as old as computing - likely 60-70 years old. The thing about it is we had the blueprints for a lot of stuff like neural networks almost at the dawn of computing, but it took almost half a century for us to even begin to try out some of the ideas, because the computing hardware wasn't even close - it would have been like trying to build a CPU out of vacuum tubes.
Once we finally had the tools to even start trying, in the late 80s/early 90s, it took us a very long time to "calibrate" these general ideas and figure out the "devils in the details" that were necessary to make certain ideas viable (for example, neural networks were discarded as a dead end in the 80s, and only considerably later were we able to discover that multi-layer networks essentially "salvaged" the idea).
Machine learning without the era of "modern computers" was a bit like flight before we'd really mastered the internal combustion engine - we understood quite a bit about it, and had theories about a lot of stuff (like the basic shape of a wing), and could successfully build gliders and such. Contrary to a lot of propaganda, the Wright Brothers didn't just arrive in the world like "lightning from a clear sky", but ... it had to become practical to do for us to then move on to putting the ideas through the paces, and all of the established theory from beforehand ran into the usual treatment of "no plan of battle survives contact with the enemy".
In terms of its origins, and the core algorithms, I agree neural networks have many decades on them.
However until the hardware and software support for mainstream massively parallel execution became available it was a niche tool.
So the level of adoption, experimentation, deployment and research resources available are multiple orders of magnitude greater than 20 or 30 years ago.
As a practitioner (for my entire career) the field still operates as a new field, with enormous areas for new experimentation and interesting new creative advances happening quickly.
Do you disagree substantively with anything I have said, or do you just think I could have phrased it better?