I don’t discount foundational math. I do a lot of DSP, and many things in audio can be very elegantly solved with math.
The point I was trying to make is that edge detectors and feature descriptors like SIFT and ORB claimed to have a nice mathematical solution when in fact they are just throwing some intuitively helpful math at an ill-defined problem with an unknown underlying probability distribution. For these problems, NNs just perform much better, and the idea that handcrafted feature descriptors have some mathematical foundation is just false. They are mathematical tricks to approximate an unknown model.
The point I was trying to make is that edge detectors and feature descriptors like SIFT and ORB claimed to have a nice mathematical solution when in fact they are just throwing some intuitively helpful math at an ill-defined problem with an unknown underlying probability distribution. For these problems, NNs just perform much better, and the idea that handcrafted feature descriptors have some mathematical foundation is just false. They are mathematical tricks to approximate an unknown model.