The problem is “where”.

Building them in random lots in Florida or upstate New York is exactly what builders were doing in the runup to the financial crisis.

Building them in already-dense places is always tricky because of NIMBYism

then it wouldn't be "smartest investment"

I moved from an expensive real estate market to a less expensive one. That has nothing to do with appreciation — just equity.

Further, when I downsize — even in the same real estate market — that too is equity working in my favor independent of appreciation.

How is getting your own money back, many years later, without appreciation "equity working in [your] favor"?

Without the appreciation (and leverage multiplying that), buying housing would be nowhere near as good an investment. (As it stands today, it's phenomenal, of course.)

The alternative of course is giving your money to a landlord (and zero equity and zero money back). That is certainly less favorable?

I'm on team "own the home you live in if your circumstances allow", but if there was no appreciation, your equity isn't "working for you" in any way that I can detect/imagine.