Sometimes I avoid rereading books for the same reason. There was a Summer where Catcher in the Rye felt very important. I'd hate to reread it under my new, adult perspective. I'd prefer to let it linger in nostalgia.
I had the same same thing with the much less erudite Stranger in a Strange Land. Re-read it 20 years later and was disappointed. On the positive side, I should consider how much I've moved forward in that time :) or sideways :-S
The nice thing about Heinlein is he has a book for all stages. As I've gotten older I really prefer his "Time Enough for Love" novel, which is about looking back on a long life.
The same thing happens to me with video games on a regular basis. Younger me simply had much lower expectations and poorer taste, back when a NES controller (with all of four buttons and a D-Pad) sufficed.
Sometimes I avoid rereading books for the same reason. There was a Summer where Catcher in the Rye felt very important. I'd hate to reread it under my new, adult perspective. I'd prefer to let it linger in nostalgia.