I lived in the south for some time, and being from the midwest ('no' accent), I was always called a yankee. Sometimes as a joke, but there were plenty of cases where it made me feel uncomfortable.