Given this I fail to understand why being a professor remains such an attractive career path.
I wouldn't say that many people see it as attractive relative to high-degree-of-freedom research positions in industry, but there just aren't that many such positions in industry. And, you often have to build up a reputation in academia first (as you did) before you can jump to an industry position that is senior enough to give you research & publication freedom. Even then there are only a handful of options; MSR and parts of Google are two of them in CS, and the numbers dwindle considerably if you look outside of CS. If MSR hired many more people straight out of grad school, and in a broader range of fields, I'm sure many grad students would consider that option rather than pursuing a tenure-track faculty position. Back in the Bell Labs days, it was a popular top choice out of grad school, with faculty job being a second choice.
If you relax the requirement that the job has to let you regularly talk about and publish the results of your research, there are more industry positions available, like R&D positions at petrochemical and aerospace firms. They have large R&D arms, but the average employee in them will publish little to none, except whatever ends up being published via patent filings. That can be a good choice (I have some acquaintances who work in R&D at BP and like their job), but a research career where you can't publish is a quite different choice of career.
As a more minor working-conditions point, I personally like the flexibility. I typically spend 3 days a week in my office and 2 days working off-campus, which most companies won't let you do. And if I want to take a 3-day weekend trip somewhere, I can just do it, as long as it doesn't interfere with a day on which I have classes; no need to ask for permission in advance or worry about how many vacation days I have.
I wouldn't say that many people see it as attractive relative to high-degree-of-freedom research positions in industry, but there just aren't that many such positions in industry. And, you often have to build up a reputation in academia first (as you did) before you can jump to an industry position that is senior enough to give you research & publication freedom. Even then there are only a handful of options; MSR and parts of Google are two of them in CS, and the numbers dwindle considerably if you look outside of CS. If MSR hired many more people straight out of grad school, and in a broader range of fields, I'm sure many grad students would consider that option rather than pursuing a tenure-track faculty position. Back in the Bell Labs days, it was a popular top choice out of grad school, with faculty job being a second choice.
If you relax the requirement that the job has to let you regularly talk about and publish the results of your research, there are more industry positions available, like R&D positions at petrochemical and aerospace firms. They have large R&D arms, but the average employee in them will publish little to none, except whatever ends up being published via patent filings. That can be a good choice (I have some acquaintances who work in R&D at BP and like their job), but a research career where you can't publish is a quite different choice of career.
As a more minor working-conditions point, I personally like the flexibility. I typically spend 3 days a week in my office and 2 days working off-campus, which most companies won't let you do. And if I want to take a 3-day weekend trip somewhere, I can just do it, as long as it doesn't interfere with a day on which I have classes; no need to ask for permission in advance or worry about how many vacation days I have.