I moved to the Bay Area in mid 2019 from another country. I've had discussions with my American colleagues where I asserted that we were overpayed for what we do. They insist that it's deserved because we're the top engineers of the world working at a top company. But I'm 100% certain the only reason we're paid as much is because of the insane cost of living in the Bay Area. I know many great engineers, or even my former-self, in my home country who are not paid even 1/3rd of what we get paid in the Bay Area. I think this WFH revolution will be a real reckoning for the software industry and the entitlement of many Bay Area engineers.
I'm sure the cost of living is a significant factor, but don't overlook regular old capitalism. The giant FAANG companies need huge quantities of programmers to grow, they are fighting with their competitors to hire those programmers, and they're all convinced (true or not) that they need only the best programmers and that their interviewing systems which can identify such programmers.
Sure, the companies tried the standard approach of just secretly agreeing not to compete with each other for talent, but the DoJ didn't care for that, and so they just do the other thing: pay as much money as is necessary to get the folks they want.
Apple makes about $435,000 in profit (not revenue) per employee, including non-technical employees. If they think that they need to pay staggering salaries to programmers to keep a good thing going, they will absolutely do it. Would they like to pay their programmers a third of what they're paying now? Totally. Would such a salary be fair? Totally. But those programmers would just call a Google or Facebook recruiter and be out the door the next day.
You're right, the real reason is the high density of companies and the talent they attract to the Bay Area. My point with cost of living was that companies need to pay $(CoL + X + Y) for people to be willing to move out here. Where X is whatever they need to pay to entice the employee away from the other companies. And Y is the amount of savings/discretionary spending the employee wants to have.
I always wondered why the FAANG companies don't move their main campuses to somewhere with a lower CoL (call it CoL') so they only need to pay $(CoL' + X' + Y) to keep the employee. Paying X' (where X' probably less than X) because they're no longer in local competition with the other companies. I wonder how long it would take to recoup the cost of moving the company in savings on compensation. There are countless places which have better quality of life than the Bay Area, while also having lower CoL. I chose to move to the Bay Area because Y was greater than what I could get anywhere else.
I agree, otherwise way more jobs would have huge compensation growth in the bay area alone if cost of living was enough to explain the income growth of engineers.
There is a reason why software is %20 of the economy and most economic growth in the USA now and why engineers are paid a lot and I believe they are correlated.
The competition of Bay Area (and California, in general) engineers is definitely a factor. But it doesn't preclude rockstar engineers existing in other areas unwilling to relocate to the region. This policy gives tech companies the best of both worlds (theoretically).