To add to what others have said, we know from psychology that the human brain works by building a (causative) model of the world, and then propagating the signal where our model goes awry and differs from what we experience.

GPT will start becoming really powerful when you just give it arms and legs and let it build it's own training dataset the way regular animals do (in the process of evolution & throughout their lives).