Well, yes, except that physicists were pretty confident that they nailed electromagnetism, except the pesky thing that the speed of light was c - but relative to what? Naturally there should have been some medium (the "ether") but when they looked for it they weren't there: the famous Michelson–Morley experiment.

So people had already known that there was some glaring gap in our knowledge, and something had to fill in the gap.