Here are stats on math. In short: 1157 PhDs given out, and 761 have academic positions (the main place to get a job as a mathematician).
Actuaries, quant finance (it's not dead yet), crypto-related programming, logistics... there are lots of non-academic jobs (most of which pay better than all but the best academic appointments). It may be the case that giving up four years of earnings is not worth it from a purely pecuniary standpoint, but I seriously doubt that there is an overabundance of math PhDs given the jobs they can take.
The private sector is, of course, much less visible than academia.
One assumes that a math PhD went to the trouble of writing a dissertation so they could do something with it. I know little about the other fields you mentioned, but a PhD isn't needed for actuarial work. A bachelor's (heavy on the statistics) with some finance courses covers you, and after that it's self-study for the seven (7!) exams you need to be a fellow.
A math PhD in an actuarial setting is out of place, like a CD grad working at Geek Squad (a hypothetical Geek Squad with pay comparable to programming).
Most of those jobs don't require a PhD. Perhaps a few of the quants, a few crypto programmers and a few operations research people. But certainly not all of them. As I noted, even many academic jobs (teaching positions) don't really need it.
While having the degree helps you get the job ("omfg you have a PhD you must be so smart!"), it isn't necessary. The number of jobs where PhD level mathematical knowledge is necessary is actually quite small. Producing more PhDs will probably not be as helpful jmtame seems to think.
Agreed. A PhD is an apprenticeship to become an academic. It doesn't actually say anything either way about how smart someone is (i.e. most people smart enough to get a good first degree could probably complete a PhD if they chose to). In industry a PhD is only of value when hiring into the R&D group (or in the case of investment banks, wanting to impress unsophisticated clients).
Plus, a reasonable fraction of the academic positions are purely teaching.
http://www.ams.org/employment/2007Survey-DG.pdf
I know physics is similar.