The question is predicting recidivism. Whether someone will commit a crime again after being released. If they are a danger to society. And grouping people into low risk or high risk categories, so people spend less time in prison, but really dangerous criminals are watched.
If human judges do better at this task, it's very easy to test it. We ask them to predict what criminals are low risk or high risk. And they barely do better than chance, and are easily beaten by the simplest statistical baseline. Yes algorithms have edge cases, but humans have far more.
If human judges do better at this task, it's very easy to test it. We ask them to predict what criminals are low risk or high risk. And they barely do better than chance, and are easily beaten by the simplest statistical baseline. Yes algorithms have edge cases, but humans have far more.