I agree that data-driven decisions are less common than they should be. However, there is such a thing as over-dependence on data, and this piece might be toying with the line. You don't have enough resources to test every little thing - sometimes you have to use your intuition and move on.
I understand that, actually, and left it out of the blog post only for brevity's sake.
When I make decisions about my websites and other projects I have in mind a picture of my audience and what they're doing. This sort of experience is the basis for most intuition. Some people are really, really good at it, too.
The right kind of experiences can be data, too, but there are two dangers: (1) not being wiling to change your mind and (2) acting off too little experience.
I just expect that any "expert" whose intuition I'm supposed to trust will be able to justify their opinion when push comes to shove.
I just expect that any "expert" whose intuition I'm supposed to trust will be able to justify their opinion when push comes to shove.
Don't expect them to justify it solely on your terms, however. If you're breaking into a new market or operating under asymmetric information pressure with significant uncertainty -- or even just regular old data starvation -- appealing to similar but difficult-to-quantify means of supporting one's intuition may be crucial.
It would be better for all concerned if the person making a claim could always articulate their reasoning in toto, but under the pressures of time, you may occasionally need to rely upon a person's track record, and trust them. Because their track record under similar conditions is a prior -- that, too, is data.
Extraordinary claims require extraordinary evidence -- but people with an extraordinary track record can sometimes be trusted to make extraordinary judgments with ordinary data.
