I wouldn't be surprised if autoregressive LLMs had some chaotic attractors if you stretch the concept to finite discrete state (tokens).