I guess I was thinking, get a random sample of newspaper issues from each of a number of newspapers, and for each article in each of them, get a subject matter expert (or, not necessarily an expert, but someone knowledgeable in the field), and give a random ordering of (a selection of) the articles in the issue, and have them read them in that order, reviewing how accurate they think the information is before going on to the next one,
And then evaluate whether, if they thought any inaccuracies they found in the article about the topic that they have a good understanding of, whether that influences the accuracy they estimate of the later articles.
Or something like that.
Maybe have them answer some other questions about the articles other than just accuracy, such as quality of writing, in order to be less likely to cause them to re-consider how they are estimating accuracy?
And I suppose by “accuracy” I mean to also include “not misleading”, in addition to just “not false”.
Or something like that.
Maybe have them answer some other questions about the articles other than just accuracy, such as quality of writing, in order to be less likely to cause them to re-consider how they are estimating accuracy?
And I suppose by “accuracy” I mean to also include “not misleading”, in addition to just “not false”.