Netflix seems to be the biggest counter case - grew their data centers and effectively gave up and moved it to AWS. I suspect the sweet spot is doing a bit of public and private cloud, adjusting how much is on one or the other based on costs, service levels and capacity requirement volatility.

Good advice but I'd argue there's one tweak to make that even better: start outside the cloud (say, just some Linux VM's from Linode or whatever), then only if you get enough real customer/visitor demand to warrant easy/virtual scaling, then move to a cloud provider. Needing a cloud/elastic hosting provider is a bit of a Maserati Problem. If you get to the point where you have to build/manage your own data centers (like Google, Amazon, Orbitz), you have a Fleet-of-Maseratis Problem.