Quality of life is usually better for those who exercise and that becomes particularly evident as people age. But either way it's literally just like a totally false tradeoff - the time I spend with people hasn't been at all compromised by exercising and you can even exercise with other people too (e.g. friends at the gym, beach, going for a run with partner etc.)

I think if you've made the choice to just not exercise you should be honest about why instead of inventing non-problems like exercise significantly reducing time spent with family and friends. I literally don't know anyone who has ever raised this as a problem. Maybe if you're exercising as an Olympic athlete...?