I was thinking about that a while ago and came to the conclusion that it's likely massively helped out by the narrow search space. They aren't trying to match between every single person, just the ones in your photo library which is an extremely small group compared to what most facial recognition is doing.
Google's face recognition can't tell the difference between my 5yo and my newborn. And, most hilarious, my 8yo could unlock my wife's iPhone with face recognition when she was 2yo.
Google Photos works in a near perfect extremely constrained closed system: your photos have fewer than a hundred faces, it likely biases uncertainties with more confidence due to those constraints.
Funny because I don't use the feature and in my review tab I have like 15 versions of myself that google thinks are all different people for me to individually name. Mostly different phases of facial hair.
