I think the prior probability in the bayesian sense is that the two entering cohorts are equally skilled (assuming students were randomly split into two sections as opposed to different sections being composed of different student bodies). If this were the case, the implication is that performance differences in standardized tests between cohorts are due to the professor (maybe one of the profs didn't cover the right material), so then normalization could be justified.
However if that prior is untrue for any reason whatsoever, the normalization would penalize higher performing cohorts (if it were a math course, maybe an engineering student dominated section vs an arts dominated cohort).
Right, and if it depends, maybe we just don't do it then?
Intuitively and in my experience, course content and exams are generally stable over many years, with only minor modifications as it evolves. Even different professors can sometimes have nearly identical exams for a given course, precisely so as to allow for better comparison.
However if that prior is untrue for any reason whatsoever, the normalization would penalize higher performing cohorts (if it were a math course, maybe an engineering student dominated section vs an arts dominated cohort).
So I guess.. it depends