You probably have a significant selection effect in play there. You would not notice the rich people you've encountered who were perfectly polite to you and did not have ostentatious displays of wealth that clearly labeled them as rich.

You may also be surprised to discover that the "rich asshole" who ducks into the spot in front of you with his Ferrari and then tells you off with his vigorously waving arms decorated with a Rolex could be entirely faking it, actually hip deep in debt and not able to afford what he's waving around. Unsurprisingly such people are often quite stressed. (I don't know "that guy" but I know some people who certainly trend in that direction and I believe would have required only small life pushes to be "that guy".)