I argue that the attention mechanism is just Hopf algebra convolution. The QK mechanism selects possible candidates and the OV mechanism picks one of these possible candidates.

I have recently written a paper on this https://arxiv.org/abs/2302.01834

I also have a discord channel https://discord.cofunctional.ai.

Ironically, it's the same mechanism as what renormalization in QFT does. I'm getting increasingly convinced that it's also how the brain works.