I do not know the people's background of a lot of comments here. They might have much more experiences than me with tensors. But, in my deep learning code and works, when I need to design an operation that involves a mix as little as 3 tensors with 4+ dimensions, I always struggle. I need to draft some slices to understand which slice should be contracted etc.. Many times the shape of the output is not even clear in my mind. Plus, add some padding maskS on the tensors and it confuses me quite a lot.
I really like this notation, the last example of 1.1 is readable in its sum formulation, but the diagram formulation is much more "alive" in my mind.
I am really lost here if I have missed something about indices notations with tensors or some visualization techniques. Or maybe the confusion of a tensor operation depends of the field? Or maybe I just miss practices and experiences with indices notations...
Standard tensor notation in pytorch and other libraries is very indirect, and often shapes are only documented in comments.
I definitely find it helps to draw parts of your architecture as a tensor diagram. Or perhaps use a library like tensorgrad which makes eveything explicit.
I am really lost here if I have missed something about indices notations with tensors or some visualization techniques. Or maybe the confusion of a tensor operation depends of the field? Or maybe I just miss practices and experiences with indices notations...