Northern manufacturing bought cotton from slave plantations, yes. As did British mills.

However, you'll note that in the absence of southern cotton during the Civil War, these textile mills did not shut down or become unprofitable, they switched to alternate sources of cotton.