And I think a more accurate rephrasing would be "I wouldn't want to accomplish what Ghandi accomplished because I don't want to suffer that much."

He's not objecting to the low social status Ghandi had, but the hardships that were necessary for him to endure to obtain what he wanted.