We don't let people sell themselves into slavery even though that would "take away the bodily autonomy of the employee to enter mutually consensual arrangement and instead smash them with third party imposition...", no "union baron" needed.
Your race to the bottom can affect others, regardless of wether or not you recognize negative externalities exist.
I don't see a compelling reason why I shouldn't be able to act as a slave for as long as both parties consent to it. If you want to create lovich 1840s era cotton picking slave service with the full historical recreation it's not clear why that's anyone else's business but you and the masta'.
That some may agree to slavery-until-i-say-uncle is far less threat to liberty than some democratic oppressor that picks and chooses what consenting adults may agree to based on what hypothetical options might end up on the negotiating table of other consenting adults.
Your race to the bottom can affect others, regardless of wether or not you recognize negative externalities exist.