I think it's a consequence of taking away individual judgement and becoming, it seems, terrified of letting a person make a decision, because sometimes they're bad at it, or have biases, or whatever. Organizations are afraid of individuals deciding things, and individuals are eager to push their decisions onto a "process" or "policy" because if they actually make a decision and anything goes wrong—whether or not the decision was a good one given the information they had—they're toast, fired if they're lucky, publicly demonized and blackballed and fined and jailed if they're not. The individual cost of a decision that is bad—even if it's only bad in hindsight—is too often way out of proportion to the harm it caused, so why risk it?
I don't know why it's happening, but it sure explains a lot of things across much of US society. Politics, business, education, parenting. I wouldn't be surprised if it's part of what's behind the frightening US "cost disease" phenomenon from the Slate Star Codex post that was discussed here a few months back.
I'm only speculating, but I think social media has only exacerbated this problem. Now every screw-up an employee makes (or a teacher, or an administrator, or whatever) is magnified times ten to everyone on the aggrieved party's Facebook feed. Have a teacher that said something a parent took offense to? An employee who misplaced an item you had ordered? An employee can always fall back on the policy and push the blame to the institution and the institution can say "we'll review our policies".
Funny thing is that as a bureaucrat, coming up with policies is my job. It tends to improve overall outcomes and satisfaction, but the same outliers described above are also magnified ten-fold (this policy is stupid because ...).
My best guess is that it's the result of 20-30 years of never accepting "shit happens, we looked and everything seems basically fine, we're not changing anything" as an answer when something bad happens and someone asks "so what are you doing to make sure this doesn't happen again?"—even when that answer's actually appropriate. The consequences are so bad if you make the do-nothing call and get it wrong (or are just unlucky) that everyone does a whole bunch of stuff to ensure they can say "look at all the things I did!" if lightning strikes twice (or if they were simply wrong and something did need to be done, but it wasn't clear until a pattern emerged)
Instead we always freak out and add a bunch of new policies/guidelines, some more mandatory educational videos or classes, and maybe bring down the hammer on some people as well.
[EDIT] in fact it may be yet another symptom of post-Nixon-era cynicism. We expect institutions to lie to us about everything (which, to be fair, they often do and often have), so we (rather, the media) dig into any problem expecting criminality and neglect and generally bad behavior (and a great story!), so institutions have to go nuts to make sure it never looks like there's anything there worth reporting as maybe-sort-of-kind-of any of those bad things.
I am by no means an expert on this subject so correct me if I'm wrong but....
The exception that I've noticed from the airline-industry, is that they actually want to solve problems systemically and don't care who has to take the blame.
For something to go wrong, multiple failures have to happen, and there's not a lot of blame put on individuals, as much as "the process".
In most other organizations, IME, fixing "the process" is expensive. It takes time, and effort, and it's quicker and easier to blame "those idiots in IT/accounting/HR/etc". Most of those types of organizations don't actually want to solve problems, they just want the problem to go away. So they end up choosing the easier option, rather than dealing with the problem systematically.
I am not an expert but I know many pilots, watch only documentaries, read research papers almost exclusively, and have stayed in a Best Western.
NTSB is, as far as I understand, actually very careful about not blaming anyone. They look for causes and solutions, not blame. They, as far as I know, pride themselves on this.
They try, again inasmuch as I know, to not blame a person - but to blame a process. This is not a distinction without difference. It fosters an attitude of cooperation and openness.
Again, I'm not an expert but I know a bunch of pilots and listen when they speak. If the NTSB blamed people, I suspect they'd have a lesser willingness to speak positively about NTSB.
Edit: Someone beat me to it. I will leave this as I think it offers a bit more of a comprehensive view.
> They try, again inasmuch as I know, to not blame a person - but to blame a process. This is not a distinction without difference. It fosters an attitude of cooperation and openness.
Because even when the cause was blatant human error, it can still be a process problem -- how did this type of human error slice through all the protections against it? What process can be put into place to prevent disaster even when some bonehead does that same thing again?
But air travel safety is atypical.
If the crash rate for planes was the same as it is for cars then air travel would have to be prohibited as a necessary measure to prevent the extinction of the human race.
Most endeavors are not of that sort.
And so in most other contexts we could use their methods to produce a process that will prevent a particular category of trouble, but suffering the trouble costs $2M (instead of $2B) and implementing the process costs $20M.
And then people will want to implement it anyway, even though they shouldn't, because "it solves the problem".
Or, seeing the obvious fallacy in spending $20M to prevent $2M in harm, a "compromise" is proposed to spend $1M to prevent 5% of the $2M harm, still with no one doing the math. And then, problem still 95% unsolved, more half measures are kludged on over time until the surrounding bureaucracy becomes politically powerful enough to be self-sustaining.
Because people don't want to admit that some diseases aren't worse than their cures.
> If the crash rate for planes was the same as it is for cars then air travel would have to be prohibited as a necessary measure to prevent the extinction of the human race.
Well, no, this is obviously false. Much more car travel is done than air travel; bringing air fatalities up to the level of car fatalities would have a negligible effect on the human population.
Bringing crashes per vehicle mile up to the level of highway crashes per vehicle mile according to https://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/pu..., assuming that an average airplane carries 140 people, and assuming (very wrongly) that crashing it kills all 140, you could get deaths to airplane crashes up to 843,000 per year for the US. This would be significant -- it would be an increase of over 30% in the annual death rate -- but it is less than the existing surplus of annual births over annual deaths. A modest reduction in population growth just isn't going to drive the human race extinct.
You're assuming the existing expense of making air travel safe is not reducing the quantity of air travel done.
Imagine it was as easy to become a pilot as to get a driver's license, there were no flight plans or restrictions on where people could take off or land and a 500MPH jet could be purchased for $150,000 because there would be less regulatory overhead and more competition in aircraft production.
Affluent people would be commuting by jet. There could easily be ten times as many air miles traveled or more.
If you want to "reason" that way, take into account that (1) if people started dying from air travel at high rates, they'd do less of it; and (2) societies with much higher death rates have no problem growing anyway, as they compensate with higher birth rates.
> if people started dying from air travel at high rates, they'd do less of it
That doesn't seem to stop people from driving cars.
> societies with much higher death rates have no problem growing anyway, as they compensate with higher birth rates.
That's just survivorship bias. Why would one cause the other? There were also societies with high death rates and low birth rates which for the obvious reason no longer exist.
> There were also societies with high death rates and low birth rates which for the obvious reason no longer exist.
If you claim to be worried about extinction of the human race, you'll have to consider all societies, not just the ones with pathologically low birth rates. A tiny group of people committing suicide has zero effect on the overall human population.
>They look for causes and solutions, not blame. They, as far as I know, pride themselves on this.
If the cause is a person's behavior, that is the same thing as blame. Look at the report of the Pinnacle 3701. Probable causes from [1]:
1. the pilots' unprofessional behavior, deviation from standard operating procedures, and poor airmanship;
2. the pilots' failure to prepare for an emergency landing in a timely manner, including communicating with air traffic controllers immediately after the emergency about the loss of both engines and the availability of landing sites;
3. the pilots' improper management of the double engine failure checklist
It's hard to look at that and say that the pilots weren't being blamed.
NTSB, i think, no one takes the blame. For example, if the error is something like "Mechanic didn't tighten bolt to appropriate torque."
So, what's wrong? maybe they forgot and need a checklist item? Maybe the reading on the wrench should be recorded in the log? should there be a second person that verifies?
They know they ask a lot of people, and it has to be right, every time. People get sick, tired, distracted, whatever. we're people. People are the least reliable part of the system.
I think blame only falls to a person when they've lied. Falsified a log, claimed to do something they didn't etc.
That's the thing. In the Air Force (at least)- there is no thought of cost/benefit analysis. A single machine fails once because a bolt wasn't torqued and causes $5000 of damage ... So they add the checklist item, and the second person, and an extra training item, and a log record that has to be maintained. The list of special rules and required documentation only ever gets longer. Eventually, you spend all your time on the checklists and maintaining the documentation, and none on actually doing whatever procedure you were originally messing with that bolt for.
As someone who was a mechanic in the Army, I feel that checklists more often helped then harmed. Many mechanical problems we saw in the shop were easily attributable to an operator skipping a step on their PMCS. Doing something relatively dangerous or confusing could be error prone, especially for people who haven't done it much, and a checklist can help ensure the safety of everyone in the shop. A good example is running an engine indoors. Before you did it, there were a serious of steps you had to walk through before you turned it on, that reduced people getting sick from carbon monoxide quite a bit.
"The Checklist Manifesto" is an -excellent- read on the helpfulness of checklists in preventing problems and mistakes (primarily from a surgeon and medical viewpoint but he covers airlines as well.)
As a submarine fire control tech, I always thought it was cumbersome to have three full size binders open in my lap during weapon firing drills, casualty drills, everything except calm open water transit. But flipping those pages and grease pencilling things even while in the hot of prepping weapons to leave the tube avoided a LOT of "hurry up and whoops!" mistakes. Sometimes crufty stuff needs a review and rewrite but, n the whole, I'd rather know what happen wrong the previous times and avoid it myself.
This kinda contradicts the idea further up that too much attention is paid to process, at the detriment of, for lack of a better term, "skill" (or "personal responsibility")
I think the focus on process can be terribly annoying, because it's so much easier to say "Bob fell asleep at the machine" than diving into all the events that led up to Bob falling asleep. Note that both can be true or, more accurately, the process problems actually need the statement "bob fell asleep" to be true to make sense.
The idea to prioritise processes is that it's repeatable. Individual behaviour isn't within your control, except for the processes that hired somebody, educated them, defined their tasks, and created the environment in which they perform these tasks.
In a small team, or family, or group of friends, it's perfectly fine to dock Bob's allowance for falling asleep. And the instinct to exact revenge, or punish people is strong, because those instincts are the result of evolution and norms of society, both of which until recently only knew small groups where everyone knew everybody. That's why it takes a lot of discipline to act against these instincts, and obviously, as you said, because focussing on the process may also uncover others who failed, and it is more tedious.
People get frustrated with bureaucracy. Sometimes they're obviously right. But the idea that all bureaucracy is wasteful, or (equally common) that governments have any particular talent for wasteful bureaucracy is somewhat misguided. Because there aren't many examples for organising teams of sometimes hundreds of thousands of people to work on shared goals without a wasteful bureaucracy, and governments and the private sector tend not to differ that much.
I've seen one interesting example of a different structure for organising a large number of people: capitalism. It arguably works, being excellent at delegating authority to exactly the right person for any possible decision. But I believe even that example isn't convincing, if you start considering all the "waste" that capitalism hides in plain sight: advertisement, competing efforts doubling work, the complete financial sector, non-monetary losses due to financial hardship, the losses from unemployment being essentially wasted potential etc.
I think it's less about outside regulators than organizations/individuals optimizing to cover their own asses. I assure you, I'm not in the "gov'ment regulators ruin everything" camp.
> I think it's less about outside regulators than organizations/individuals optimizing to cover their own asses. I assure you, I'm not in the "gov'ment regulators ruin everything" camp.
The problem being that the ass-covering is a predictable response to known stimuli.
What do you propose to do to get them to stop doing that?
I believe you're being slightly unfair to those institutions. As one example: the US Government, as a policy, never lies to the American press (and, by extension, the people). They may sometimes do so to the foreign press, or they'll deny comment, or they'll give you an answer that doesn't actually answer your question. But there used to be a line they dared not to cross.
There are many reasons for this. Some will insist it's the morally right thing to do. But a better explanation is usually that it's in no one's interest: Career bureaucrats are simply never in a position where lying is a good option unless they're trying to cover up a crime they committed. They're risking their job, and possibly criminal prosecution, without any real potential upside. For elected officials, the same is usually true–with the potential exception of a lame-duck President who has nowhere to go but retirement. But at that point, they've probably gotten used to managing their office without lies.
Of course that's a description of the past, and the lower reaches of elected executive office (but not employees) have a worse track record, where many a town mayor with a coke habit has lied to cover his creative accounting. Nixon also comes to mind. But before you say "Iraqi WMDs", I'd say that was arguably a case of a government convincing themselves through wishful thinking.
Maybe they're just really good at lying? It's the old problem: is the revelation of a scandal proof of spreading corruption, or does it show that the system is working, and the bad apples are caught? It really seems to come down to personal judgement. I can only say that having interacted with governments on multiple levels, and seen a lot of, for example, the vast amounts of data they produce and decisions they make that are never the subject of the news, I have come away with the impression that the Government is usually trustworthy, and that decisions are often made with a surprising lack of emotions. For example, I once won a lawsuit against a local government. Yet a few months later, we won a tender that we deserved to win, even though the decision was made by the very same people we had previously fought in court.
If all else fails: even if government were corrupt to its core, saying so would be less effective than acting as if it weren't, and focussing on specific incidents. Because faced with a public that believes only the worst of every politician removes all incentives for them not to become exactly that sort of thoroughly corrupt politician. And if it's part of society's lore that all powerful people lie, that becomes a self-fulfilling prophecy when people new to government just follow the clues they got from popular culture.
I agree other countries are worse but the US government makes things up and lies constantly. Obama wasn't perfect, but I don't think it's partisan to point out how Trump's admin is utterly blatant in this, comically so.
If you accept that the US Government didn't lie just told the least untrue thing it could then it fits. It just turn outs the former is identical to the latter in most cases.
I think you meant to say "US Government, as a policy, never tells the truth". If they did, we the people would have thrown them out on their asses before their terms are up.
Yeah, I think the big problem is trying to come up with the one big issue and change that, as if that will not have a whole bunch of issues.
People want one answer on what the problem is, because then the most amount of people can understand it in a tweet. They don't want complicated solutions. You say it yourself, it is one problem.
I think you're observing Pournelle's Iron Law of Bureaucracy, which I guess could be stated pithily as "The goal of every bureaucracy is to become self-sustaining. Those that don't are replaced by those that do":
It's interesting that another active comment thread on the HN front page today is about evolutionary game theory & the selfish gene. I'd posit that there's a whole class of "evolutionary systems", subject to the axioms of 1.) large numbers 2.) random variation 3.) selection bias and 4.) competitive pressure. And any such system will always tend toward survival and maintenance of the status quo as its primary imperative, for the simple, tautological reason that those organizations that don't are replaced by those that do. This operates on multiple levels, from the molecule to the gene to the cell to the individual to the firm to the bureaucracy to the society as a whole. Oftentimes, behavior that makes no sense on one level seems obvious if you consider the "actor" to be the level above or below what you're looking at: the gene instead of the individual, or the firm instead of the individual, or the society instead of the firm.
In your example, for instance, the primary actor whose survival that the bureaucracy optimizes for isn't the individual: it's the bureaucracy itself. Individuals who serve their own interests instead of the bureaucracy are rejected and fired, as are individuals who serve the other individuals that the bureaucracy supposedly benefits. Why? Because bureaucracies without this incentive structure are replaced by bureaucracies with it.
And there actually are plenty of areas within U.S. society where bureaucracies are not the primary actors. The individual is still valued over the firm or bureaucracy within therapy culture, artist communities, fandom, Internet forums, dinner parties, and many other social contexts. But because there is no central bureaucracy or organization to point to, these niches fall off our radar screen; niches where individual humans remain dominant are by definition areas without enough resources to support a bureaucracy.
Baumol's cost disease is different, and IMHO a more short-term ailment that's afflicting developed economies as they transition to the information age. This phenomena of growing ever more complex systems on top of individuals is pervasive, and will continue to happen as long as the population keeps growing.
The trouble with that analysis is that "self-sustaining" doesn't inherently imply large or bureaucratic or inefficient. It's possible for a large inefficient bureaucracy to be replaced by a more efficient self-sustaining system with less centralization.
The bureaucracy can fight but it can also lose. Blockbuster Video is no longer a thing that exists because of market pressure. Standard Oil was broken up because of political pressure. There have existed countries that, following the conclusion of a major war, actually disbanded the majority of their military forces, both voluntarily and due to external force. Countries have revolutions where the revolutionaries win and then exile or execute the previous ruling bureaucrats.
In theory the same thing can happen through the democratic process, although this is obviously less common because bureaucracies have a solid understanding of how to subvert electoral opposition.
Your examples are of bureaucracies destroyed from the outside, by stronger competitors for the same resources. It's quite impossible to curb a bureaucracy from the inside, e.g. through "democratic process". The system will not fix the problems with the system.
I suppose it's like cows that evolve ever larger horns to compete for mates, then one day a new predator comes in and kills all the cows because their horns are too heavy and they can't run fast enough.
I think that's slightly different, an "outside of context" problem, as Banks would say. The cows didn't leave enough flexibility to adapt to the unexpected, rather than just being inefficient.
A less cynical take on the taking away of individual judgement is that it arises from the need for individuals to be replaceable rather than solely as a means of deflecting blame ...
Coastguard ships for example cycle the entire crew every two years. I imagine it must be similar in the navy. There is very little institutional memory retention -- and so the "process" emerges to try to direct all the human cogs as they pass through ...
UK experience here. All military units (including ships) experience staffing ‘churn’ as individuals join and leave for career and other reasons. Some military units are periodically completely reset (e.g. a new ship or a ship that has been extensive refitted), and they then have extensive training regimes to become ready for operations again. The churn then starts again. Some units persist for long periods of time (e.g. a formed air squadron) and do not get ‘reset’ under usual circumstances, although they do have churn. Others (e.g. army formations) have repeating multi-year cycles of training, being ready for ops, and other tasks.
The institutional memory is notionally preserved by the instructors in the schools and training centres (as well as documented Standard Operating Procedures, standing orders, etc). New joiners will have undergone standardized training, and although they still have to establish their role (storming, forming, norming, etc) when they join an existing team, they do in principle understand what they are supposed to be doing. In some cases, new joiners bring valuable new knowledge, skills or attitudes, so churn isn’t always bad. Individuals who have joined up for the long term will themselves have cyclic periods of training, with initial trade training followed by time in a unit, then additional specialised training at a school, then more operational tours, etc. Some highly-trained individuals may be embedded in the units themselves (e.g. at a barracks) to provide standardized, decentralised on-the-job training / supervision to newbies who have joined from a military trade school.
More importantly, ALL military needs to be ready for massive war. If another world war breaks out they need to quickly scale to 100x as many humans. That cannot be done unless everything is easy to teach. Follow a checklist exactly is easy to teach.
It seems interesting to contrast this with the explanation of the financial crises that decision makers had 'privatized benefits and externalized costs' (I remember hearing it from Nassim Taleb, but there were probably others making the same point)
aligning incentives between organizations and the people who make them up is hard
> Organizations are afraid of individuals deciding things, and individuals are eager to push their decisions onto a "process" or "policy"
But! but we're in IT and the Agile Manifesto is the exact reaction to that! We have a living proof that "another way" is possible. Hire champions and trust them on the task, it makes them more enterprising and crafters and makes them deliver systematic higher quality. Remember how IBM's IT used to be: Pathetic chain of specification-driven documents that "decision-makers" couldn't possibly read.
Let's just say for the moment that the rest of the industries haven't found an "Agile metgodology" that befits them. Let's hope it's just a matter of time before government-sized organizations have an epiphany about how to apply Agile to death-risking processes.
Looking at this issue more broadly, I definitely agree that non-IT industries need to take into account Agile methodology to improve their results.
I've been looking for the reason why IT has come up with Agile before anyone else (I don't consider Lean Industrial Management as Agile whatsoever) and I simply think it's because the first industry with so much information in it (it's in the title even...).
Now that other industries have to orchestrate huge processes with huge amounts of information, I hope they'll take into account what IT brought up to solve this issue instead of taking the "IBM route" further (more policies, more guidelines, more bureaucracy).
> Looking at this issue more broadly, I definitely agree that non-IT industries need to take into account Agile methodology to improve their results.
Non-IT industries are often ahead of the “Agile” movement in IT, having adopted Lean methodology and related approaches that actually applying engineering approaches to solve the problem identified, but not actionably addressed, in the Agile Manifesto.
Most so-called Agile methodologies (the exceptions mostly being adaptations from Lean methodologies, either in other industries or from the Lean Software Development movement) fail to do this, and Agile has largely become exactly the kind of cargo cult, situation-blind consultant-driven, value-blind nightmare that the Agile Manifesto was a response against.
I don't know why it's happening, but it sure explains a lot of things across much of US society. Politics, business, education, parenting. I wouldn't be surprised if it's part of what's behind the frightening US "cost disease" phenomenon from the Slate Star Codex post that was discussed here a few months back.