1) I'm not joking. I think the OP legitimately didn't think about the humans that don't look like they expect them to look. That's a huge problem across all of tech - not just our lack of imagination about the wide array of human experience, but us assuming that we have thought of everyone when we clearly haven't
2) I think this part gets trickier
How do you tell an AI not to do irreversible things? How do you indicate that this is a plastic bag blowing across the street and it needn't cause a traffic jam, but that is a plastic tarp hanging off a barricade that we can't drive through?
How do I indicate that this is a piece of cardboard we can drive over but that is a human person wearing a Halloween costume made of cardboard?
It's not an easy task, and I'm very sympathetic to the very real technical challenge they're facing
2) I think this part gets trickier How do you tell an AI not to do irreversible things? How do you indicate that this is a plastic bag blowing across the street and it needn't cause a traffic jam, but that is a plastic tarp hanging off a barricade that we can't drive through?
How do I indicate that this is a piece of cardboard we can drive over but that is a human person wearing a Halloween costume made of cardboard?
It's not an easy task, and I'm very sympathetic to the very real technical challenge they're facing