Thanks, just skimmed the paper, I think that "they automated RLHF" statement is maybe too strong here, there is still manual process but it seems like they optimized away a lot of manual labeling work.