One of my acquaintances wrote this piece a couple of years back, asking if Meta/FB's use of product engagement metrics as a signal for IC performance was a case of the McNamara fallacy:
I see that a lot in hiring. We can measure grades and experience, we can count diplomas or certifications, so we focus on these things and disregard soft skills like the ability to work with other people.
It’s not a black and white thing. You can increase the likelihood of finding someone suitable for the work by various means. For example, if someone who will actually work with the person closely is involved in the hiring process you increase the likelihood of spotting someone who might not be a good fit.
It’s a tricky topic and for sure not a binary yes/no thing, but giving up on it because it can’t be done is the wrong response (and also what the article is about)
It's only a fallacy if you have garbage or incomplete data.
Someone with poor reading comprehension will read this as "trusting the data over a more holistic approach", when the actual criticism would be something like "making decisions based on metrics than can be easily measured, rather than ALL metrics is a mistake".
Generally speaking, if you have comprehensive and complete data and domain competence, it is not a mistake to make a decision based on objective metrics.
Any real-world situation (outside of simple trivial matters)? War, health care, immigration, economics, running a business, those kind of things. Chess is just a game and not very important. No one is talking about games here.
Getting a good overview of what happens at scale is exceedingly hard, and on pretty much all of this you need to take in more inputs than what can be strictly quantified.
No one is moving anything. Does the linked article talk about chess? Or matters like war, business, health care, and hiring? The context of what's being discussed here is political and business decisions, not games like chess.
No one in this entire thread claimed that having more data is bad. This is also not what the fallacy is about and no one is against having good quantifiable data.
Incorrect! I don't know either of you, but let's play fairly.
The other person was asked if they ever had complete information, then they provided an example. Then the goalposts were moved to not only "real world situation," but "non-trivial."
It's possible to deem any real-world situation they offered in return with complete information to be "trivial," thus moving the goalposts again.
No, that's exactly what the McNamara fallacy is about.
"US Air Force Brigadier General Edward Lansdale reportedly told McNamara, who was trying to develop a list of metrics to allow him to scientifically follow the progress of the war, that he was not considering the feelings of the common rural Vietnamese people. McNamara wrote it down on his list in pencil, then erased it and told Lansdale that he could not measure it, so it must not be important."
The problem, for the US involvement in Vietnam, was that "body count" was irrelevant. The only metric that mattered was the willingness to fight of communist Vietnam, China, and the USSR. The problem is that "will" is not numerical. It is willingness to fight among the political elite of those countries and they include domestic factors, economic factors, and military factors. If anything, McNamara's fallacy is like comparing the iPhone and Android and saying the only thing that matters is screen size.
