No, it seems certain mathematicians simply assume this to be the case without any scientific argument behind it.

Perhaps it's just too easy to call something "racism" just because we don't like it?