I would argue that linear regression is under the umbrella of ML. And probably the most important ML technique for existing businesses. A lot of the "low hanging fruit" for existing business cases is not necessarily discovering how to use regression analysis on existing data (which is probably already happening at a lot of places) but how to operationalize and continually update linear regression models so you can make them a mission-critical part of the infrastructure.
Linear regression has been around since before computers. It's a real stretch to put that under the umbrella of AI. Nor is AI required to continually update linear regression models; a "for" loop will suffice.
That loop has to run over high quality, relevant data. The regression model needs to be retrained if the production data drifts from the original training dataset. The output of the regression model needs to be wired into a downstream application or decision support tool. That downstream application needs to know how to deal with error bars. A data scientist / statistician needs to re-architect the whole thing if the data or business requirement changes substantially.
As usual, the hardest problems are outside of the code.
I agree with you here Data and measurement is the single most important part of the process.
From my experience working in an industrial plant which has been involved in several machine learning trials a lot of the time there are attempts made to use complex modeling techniques to make up for a lack of measurements.
Something I question is whether the outcome would have been better if the money which was invested into hiring AI consultants was spent on better plant instrumentation.
Industrial Instruments are not cheap something like a PGNAA analyser (https://en.wikipedia.org/wiki/Prompt_gamma_neutron_activatio...) is an expensive capital purchase and I suspect some people have unrealistic expectations that AI and machine learning can replace things like this.
I think there is some middle ground where AI complements better sensors (maybe instrument companies should be pushing this). I've yet to see any of the data experts push back and say something like "actually you need to measure this better first before we can model it."
Most ML algorithms boil down to linear regression with some loss function. Even deep learning networks are essentially just linear regressions stacked on top of each other, with lower layers being trained on features predicted by higher layers.
I think if neural networks or SVMs are AI, then linear regression is as well. Neural Turing machines and other recent developments I think are closer to the layperson's idea of "AI," though.
>Most ML algorithms boil down to linear regression
I think (particularly with DL) it would probably be more accurate to claim it boils down to nonlinear logistic regression rather than linear regression. To your point, both are relatively old techniques
Nonlinear regression is still linear regression with transformed features. Kernel regression, for example, just uses features generated from the data using a supplied kernel function/covariance function. DL just allows the features to be trained.
