I know they are foundations of mathematics. They are elements of mathematical logic. No disagreement on this.

But how they are not mathematics? They are part of mathematical logic and mathematical logic or logic in general is branch of mathematics. Do you want to say they are somehow separated from math?