I'd argue it is unreasonable given that (afaik) no one is marketing a self-driving car that doesn't also require a driver be attentive at all times.
You can learn to drive a self-driving car defensively, in the same way you learn to drive a normal car defensively. Pay attention to your surroundings, blind spots, and other cars.
And Uber's and Tesla's accidents have shown us that you can't rely on the human in the car to catch it before it does something stupid. After all, teaching someone how to drive and watching them attentively is difficult work.
It's especially bad because us humans supervising the cars are going to be worried about preventing human-style accidents. Maybe you're worried about it following too close or not noticing the red light - so you don't notice the warning signs in time when it veers into the concrete barrier.
You can learn to drive a self-driving car defensively, in the same way you learn to drive a normal car defensively. Pay attention to your surroundings, blind spots, and other cars.
Maybe self-driving is a poor term to use.