Poor people are probably using public transit, not driving in the first place. Investment in public transportation would be good for them. Investment in roads (automobile infrastructure) benefits only those who can afford cars.
What you say may be true in US cities, or in the countryside.
For me, living in Tokyo, it is really cheaper not owning a car but renting it when I need to, or hiring a delivery service. I guess that the same happens in most major European cities.
As usual, this plan is OK for the more affluent and makes life more difficult for the poor.