You don't need to get everyone to coordinate. You need politicians to not listen to the lobby of home owners and real estate companies worried about their investments (in other words, ignore the NIMBYs). Change the zoning laws, incentivize developing mixed use, prioritize pedestrian and cycling infrastructure and stop prioritizing cars and parking.
Walkable cities are actually illegal in much of the US due to zoning laws right now. The reason you don't see shops in residential suburban developments is not because there is no demand, it's because it's literally illegal.
Having walkable and bikeable destinations is compatible with back yards. It just needs to be legal to build it.
In general, I believe that it's the highway lobby who shaped the US into what it is today, not the NIMBY people. I think they have much more power than nimby, even if in certain situations, they seem to be on the same side. So what I say is that industry interest weighs more than voter interest.