If that you could care less is the most you can say about something, that's an insult. If you say that you couldn't care less, that's just a lie - you're responding to it.

http://en.wikipedia.org/wiki/Damn_with_faint_praise

I love David Mitchell, but this rant by other people is a long held irritation of mine: false pedantry:)