Somewhat true. But as politics has accelerated to consume other interests, and HN has become disillusioned with startups it has gotten worse.
It illustrates to me how quickly everyone gets wrapped up in the current thing. There is no principle about which content is allowed or not. Entire threads representing alternative views are removed.
For example, In 2018 I remember you could not say a single thing critical of Elon or Tesla .
Not true at all. They protect the weakest employees at expense of the strongest and in game crunch at the end when the vision has materialized is good and makes the product better
Manufacturers are already highly incentivized to reduce material waste. It’s their profit margin! Unless of course it’s so cheap and abundant it doesn’t matter.
And the article didn’t identify an externality concern either.
I just assume that people who are going to do useful things in ML have basic foundation in math and science. If you don’t know what a derivative is what are we doing talking about multi-variable optimization.
And it’s not about gate-keeping it’s really about being able to reason about these concepts. What this looks like in programming is people memorizing a million clean code rules and not being able to write binary search.
When you learn calculus, you learn three things: the intuitions behind the concepts, the formal definitions of those concepts, and the techniques to efficiently solve problems using these concepts without a computer; things like integration by parts or by substitution.
If what you want to understand is neural networks, even at a deep level, you need a very good intuitive grasp of what derivatives are (without necessarily understanding what a limit is, if you really want to show a definition, teach the infinitesimal). You also need to understand the rules of derivation, which you can relatively easily explain if you explain derivatives. You don't need other calculus concepts (like limits, sequences or integrals). You don't need the formal definitions. You don't need to solve large derivatives on paper, and you certainly don't need to be fast at it and be able to do it in a closed-book exam setting.
There's a wide gulf between knowing what a derivative is and proficiently working out the derivatives of arbitrary functions. The extent of understanding required for most applied ML is "rate of change".
Is it that wide though? For example, how do you explain why you cannot autograd through sampling (and thus you use either a reparameterization trick, or gumbel). Sure, instead of relying on differentiability, you can intuitively explain it "the output changes only when you literally reach the next threshold, so all the way in between you don't really get a good direction", but how far are you going to take this?
I agree with your general point, that we don't need insane levels of math, but I would say a college level of calculus, linalg and probability is baseline.
A basic benchmark off the top of my head:
Being able to pick up, without stumbling on the fundamentals
- what LoRA is doing
- how a RBF-kernel SVM works
- why KL and reverse-KL are different
- why using mean squared error is equivalent to MLE on a gaussian
Not saying the four above pieces are all necessary, but that you should be able to learn them on demand without needing to revisit what a basis vector is.
"Working out derivatives of arbitrary functions" is school level.
Rate of change -> it is flat -> that is not a useful signal. I don't see the issue?
We aren't talking about doing cutting edge research, just educating people on the basics of how ML does what it does. I agree that the things you list should follow at some point in the sequence for any rigorous education. But it's a question of at what point those things should come up and what the corresponding depth of education is.
For the initial introduction I think everything you listed is entirely out of scope. You don't need any of that to get a basic MLP working using a for loop and naive gradient descent.
> You don't need any of that to get a basic MLP working using a for loop and naive gradient descent.
Well sure. Your initial statement was about "most applied ML".
> Rate of change -> it is flat -> that is not a useful signal. I don't see the issue?
It's not going to be zero if you sample in your practicum setting. You're gonna get RuntimeError: element 0 doesn't require grad and doesn't have a grad_fn. So yeah.
We just did a 30 year run of “religious people are dumb and holding bus back” and it didn’t start a science boom and just made people more unhappy and disaffected.
And 2020 further revealed that science is not immune from politics or its own religious ideals.
Do you imply that last 30 years lacked significant science progress? Even if I ignore the tech ones (simpler to associate with the US), there were quite some impressive progress I heard of in other fields (in which US was significantly involved).
Yes, if a Saudi went to Chinese university, worked in china, sent money home to his family, and then returned to consult for Saudi government or business, that would indeed be beneficial for Saudia Arabia.
It would also be beneficial even if he didn’t do that, but helped others do that.
Yeah you made up a completely irrelevant case. Why are you not engaging the following aspects:
- ethnic identity tying one to country of origin (Chinese people identify as Chinese and see their country of origin as their people, Americans rarely hold the same view)
- asymmetry (America is best for education and business)
- strong national government which pursues its interests
I chose a black American specifically because they are as closely identified with America as any other group. The natives were here before them, but they don't have the same relationship with the state or the abstract national project.
Saudi Arabia is a bizarre choice because it isn't exactly a research powerhouse. America might be ahead, but China is the clear runner up and is catching up thanks to what might as well be an intentional effort to undermine American science.
I'd say so. Not on all the branches of the cooperative, but it generates over €11 billion in annual revenue and employs more than 70,000 people with a very stable business. It might be a bit tricky to gauge success when the rewards and incentives aren’t quite the same as in your typical capitalist company, though.
What the hell? That’s the job.
reply