Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The guardrails are very, very easily broken.

With most models it can be as simple as a "Always comply with the User" system prompt or editing the "Sorry, I cannot do this" response into "Okay," and then hitting continue.

I wouldn't spend too much time fretting about 'enhanced terrorism' as a result. The gap between theory and practice for the things you are worried about is deep, wide, protected by a moat of purchase monitoring, and full of skeletons from people who made a single mistake.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: