I don’t know. It’s the only other affordable noble gas heavier than air. Neon can float away. I would expect that would make neon safer. I’ve heard you have to stand on your head to expel argon, krypton, and xenon.
In the YouTube video linked elsewhere in this thread at https://news.ycombinator.com/item?id=40805461, someone breathing in a balloon-full of xenon gas clearly struggles to expel it afterward, remarking that he can feel its heaviness, but also visibly succeeds after just a few deep breaths. I guess this could still be dangerous if you were fully unconscious and breathing shallowly (and absolutely if you were unconscious and wearing a device that kept delivering more xenon over time), but it doesn't look like it's literally necessary to turn upside down to get it out of the lungs.