Who cares? He said it. The words are his responsibility. If his speech had advocated for grinding orphans into a nutritious paste, we wouldn’t be defending him on the basis that he didn’t write those words. He chose to read them and give them his official backing.
Because the book is plastered with the author's as well as the publisher's name. Their separation is easily comprehensible. Whereas when an orator delivers, the separation of the writer is not so apparent. It is automatically assumed the orator is the writer.
This is a perfect example of a motte and bailey. The "motte" is that people should be judged badly for parroting horrible ideas they heard (which makes sense) and the "bailey" is that people should be praised just for parroting nice things they heard (which doesn't make sense).