The latest extensions from ARM are codenamed Helium, and they are an extension on the previously mentioned NEON. Both NEON and Helium are quite simple vector extensions, and yes it is also used for classic DSP stuff. I believe Helium also supports fp16, though for inference on MCUs I believe that int8 will continue to dominate. Here is book on Helium from ARM that seems informational https://github.com/arm-university/Arm-Helium-Technology

There is another chip that is generally available, that has a CNN accelerator/co-processor - the MAX78000 https://www.embedded.com/hardware-conversion-of-convolutiona...