In practice it just means that the processor includes a matrix multiply unit somewhere.

For example, some of the latest Intel Xeon CPUs have 8x 1KB registers that can perform only one operation: matrix multiply (and accumulate results). See: https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions

Generally this is very similar to the Vista-era Microsoft requirements for PC manufacturers, where it was a logo requirement to include a GPU that could be used to accelerate composition in the desktop window manager, such as transparency effects and blur. (Prior to that, low-end PCs had "graphics cards" that were fixed pipeline and not programmable /general purpose.)

Now Microsoft is forcing PC manufacturers to do the same kind of thing, but instead of a GPU it's now an "NPU" that they have to include. This can be a CPU instruction set, a co-processor, or a GPU baseline capability. The requirement is 40 teraoperations per second.

IMHO, 40 TOPS is way too low, and doesn't focus enough on memory bandwidth. Also, that 40 is total across CPU+GPU+NPU, which means in practice it'll require fiddly optimisation to get anywhere near that level of performance.

Windows Vista had the same issue, where many low-end laptops especially would struggle and spin up their GPU fans just from desktop workloads, let alone gaming...

"40 TOPS is way too low", I'm curious about this, from what I've read, Apple's M3 maxes out at 18 TOPS, and the newer M4 at 38 TOPS, so to me it sounds like even the entry-level Copilot+ PC is going to beat Apple's M3/M4 family. Am I misunderstanding.

It's possible that Microsoft's target is too low and Apple's performance is also too low. However when you find yourself saying that the entire industry is wrong you might want to stop and think.

An NVIDIA 4070 puts out 836 "AI TOPS" using 200 watts of power. So the 40 TOPS target is about 10 watts of power draw equivalent, assuming it uses a similar silicon logic tech to what NVIDIA used. With a more modern process, this is about 5 watts.

For an ultra-mobile tablet or laptop, this is... reasonable, I suppose.

For a desktop, it's quite a bit behind the current-gen tech, let alone "the future".

Like I was saying, this is aiming for where the puck was, not where the puck will be.

It's the same thing as Vista GPU requirements. Vendors will do the bare minimum that ticks the checkbox, but in practice it'll be useless garbage.

Modern AIs require > 1 TB/s memory bandwidth, > 128 GB memory capacity, and > 1,000 TOPS of compute to be really usable locally, not just technically capable of "running".

The problem is that "AI" isn't a feature. It's not a product. It's a pretty broad category of technologies that can be used to build features and products.

"AI infused at every layer" is like saying "we build our software with the best Agile(tm) practices". The customer doesn't care how you built the software, the customer cares what your software will do for them, and all "AI" means is "we probably have a deep learning inference engine somewhere under the hood, but no promises".

That's like saying audio playback isn't a feature or a product.

No, it's more like saying "media" isn't a feature or a product. It's not. It's a very vague word that could mean any number of loosely related things that do form part of many different features and products.

And that was totally a thing in the early 2000's

Companies were selling "Media PC's", and Windows ME stood for Millennium Edition or Media Edition depending on who you asked.

And people think the word "content" is overused now...

or, like saying "media ready"

I think it means many of us will not use Windows ever again.

I was already there when Windows 11 came along and blocked my perfectly functional 4 or 5 yr old HW from running it (even though the pre-releases ran just fine).

This "AI" bullshit is just another nail in the already nailed shut coffin.

Realistically, this is an announcement that Windows 12 will have a double click gguf runner and requisite backend as a feature.

Onnx, which seems to be awq

This is the big thing I’m worried about with the AI revolution. If AI is being baked into your OS, where does usable training data end and your sensitive data start? Recall is pinkie promising that it won’t record sensitive data but at the same time it’s also saying “if your application doesn’t protect it well we mayyy see it and record it, whoops”. I can bet you 120% Recall is just a way for Microsoft to collect training data on millions of users every day. It is a privacy nightmare. But I don’t think the average consumer will care. Privacy died a long time ago.

We don’t want that. Investors want that.

it's a "fundamental transformation of the OS itself", you see

Probably true. The new win64 API will have just one entry point:

  LPSTR do_thing(LPSTR prompt);

E.g.:

  if (do_thing("get a mutex or whatever it's called") == do_thing("What's the happy return code for getting a mutex?") { ... }

Someone someday may actually ship code like this into production. Horrifying to think about. For some reason this reminds me of trying to grow plants with Brawndo https://www.youtube.com/watch?v=kAqIJZeeXEc