There is another factor to consider. The stakes of asking an AI about a taboo topic are generally considered to be very low. The number of people who have asked ChatGPT something like "how to make a nuclear bomb" should not be an indication of the number of people seriously considering doing that.
That’s an extreme example where it’s clear to the vast majority of people asking the question that they probably do not have the means to make one. I think it’s more likely that real world actions come out of the question ‘how do I approach my neighbour about their barking dogs’ at a far higher rate. Suicide is somewhere between the two, but probably closer to the latter than the former.
