The ideal cost is as low as possible without sacrificing future scaling needs or development velocity. At some point, you'll start to experience diminishing returns.
It usually goes POC->Cloud provider->Your own gear
apart from the bigger companies (like Fb), have there been any major SaaS companies (esp. B2B but I guess that's trying to be too narrow so B2B & B2C) that have moved from Cloud to their own Data Center? (I know Etsy comes to mind...but apart from them?).
Hacker News and Pinboard (acq. Delicious) run on a single server.
It's not hard, but you do need to know what you're doing and have resources to do it (most orgs rent colo space in someone else's datacenter, they don't build their own). There's a reason AWS margins are so high (which leaves a lot of cost savings to be had when your workload isn't highly variable). Any questions, email is in my profile. I spent ~16 years building data centers, hosting environments, infrastructure, etc.
Many SaaS companies not only lack the ops experience needed to run their own infrastructure (which may be only problem of perception, for me running stuff on dedicated HW in colo center seems like less hassle than dealing with things like AWS) but also the common sense required to not over-engineer the scalability of their solution in the early phases (which falls squarely into the YAGNI teritory, as you can run surprisingly large stuff on two or three physical low-end 1U servers).
It usually goes POC->Cloud provider->Your own gear