That ignores the co-evolution between throat anatomy and brain structures related to language. There's no reason to select for language ability in the brain over evolutionary time if the throat can't use it. If mere ability to use sign language was enough to push language development, chimps would do much better with human-invented sign languages. Deaf and/or mute humans are, I presume, co-opting language skill "meant" (selected) for vocal language.