The problem with Llama 2 chat versions is that they have been RLHF-ed to death. You can't ask questions without getting a sermon of how your question may be inappropriate for this or that reason.
I think it's worse on the smaller models, but still present in the 70B one.
Apologies if you’d already seen this and were only trying to make a point, but you might like this article from a week or 2 ago that talks about how to run Llama 2 “uncensored” locally, and it seems to do a decent job of mitigating the sermons!
When you encounter "uncensored" in a llama model (1 or 2) what that means in that context is that the fine-tuning datasets used have had all refusals to respond removed. There's no way to uncensor the pre-trained model itself and fine-tuning only changes the style of the output.
For sure, that's a good reason for using the uncensored fine-tuned versions. There are other good reasons too like expanded context size, codegen, and story writing/rp. Just be careful of extraordinary benchmarks.
Btw, have you tried changing the default Llama 2 chat prompt? Meta tried to fine-tune it so that if you remove the safety part from the prompt, safety won't be applied[1]. Not sure how well it works myself, but worth a shot I guess
I think it's worse on the smaller models, but still present in the 70B one.