Except the number you get this way is not between 3 and 4. It's essentially infinite with no functions in the model that separate it from the finite numbers (and this can be proven in PA: if a number isn't 0..N, then it's greater than N).

Generally, I don't think it's fair to characterize this story as something studied by mathematicians. Really, it's an exercise in reasoning about nonsense.

Depends what your definition of < is. Some definitions leave unspecified what the relationship between two values not in {0, 1, 2, ...} is.

Inequality can be defined in Peano arithmetic in a total way. Maybe you're talking about defining some relation on the model? But what justifies calling such a relation an inequality?