All the time, and unless the images are next to each other you don't really notice unless you're looking for it. Not because there isn't a big difference, but because you just don't really care that much unless it is called to your attention.

I've found pixel-count improvements stop mattering for me somewhere around DVD quality for a lot of content. I can tell the difference between that and 1080P, but stop caring once I'm actually watching the show/film. For the occasional really beautifully-shot film or some very detailed action movies, I guess I might care a little. 1080P versus 4K, I don't notice the difference at all unless I'm deliberately looking for it. And that's even with a 55" screen at a fairly close viewing distance.

What does make a difference? Surround sound. I'd take 5.1 or up with DVD-quality picture over 4K with stereo any day, no hesitation.