Its weird people keep repeating that. I've been fortunate enough to meet a fair number of programmers who have moved onto make-AI-go-brrrr jobs, most of what they're handling is AMD hardware.
Nvidia strongly missed the boat by continually pushing CUDA lock in, while being on the Khronos steering committee and having made important contributions to OpenGL, OpenCL, Vulkan, and the SPIR-V ecosystem while simultaneously having pretty poor support for standard APIs in their software stack.
Highest perf per watt and perf per dollar is AMD land. AMD keeps moving fowards while Nvidia keeps making weird missteps. There is a reason why I said series 10 is the best they will ever make, there will never be a return to that: they're stuck in the same loop Raja was: make everything bigger for the sake of bigger, instead of making actual performance improving changes.
I actually work in the field (for quite a few years by now) and basically everyone I know (who knows what they're doing and isn't just getting started) uses Nvidia. AMD drivers for GPGPU compute sucked so bad for so long, noone even bothered to try them anymore. CUDA might be closed source, but it is lightyears ahead in stability and usability. If you want to get things done in AI, you buy Nvidia. Everyone who actually has to make these decisions knows that.
They are optimal in the overall sense, because flops per dollar is not everything in the real world. Though I can see why many people outside the field would think so, since that is what manufacturers write in big letters on their marketing brochures.
You are partially right. Flops isn't everything, its just a major component of your capex and/or opex if you rely on enterprise compute for the majority of what you do (ie, machine learning).
The cost of doing business that isn't flops is the realm of developer time. Nvidia tooling is where developer time goes to die. People who keep claiming Nvidia's tooling is great for developers and easy to use is someone who has a skill mismatch for the industry, or worse, someone trying to sell you something.
Nvidia strongly missed the boat by continually pushing CUDA lock in, while being on the Khronos steering committee and having made important contributions to OpenGL, OpenCL, Vulkan, and the SPIR-V ecosystem while simultaneously having pretty poor support for standard APIs in their software stack.
Highest perf per watt and perf per dollar is AMD land. AMD keeps moving fowards while Nvidia keeps making weird missteps. There is a reason why I said series 10 is the best they will ever make, there will never be a return to that: they're stuck in the same loop Raja was: make everything bigger for the sake of bigger, instead of making actual performance improving changes.