Presumably, the identified phrase list could be used to finetune a Bert model or similar that could catch more cases, as a binary classifier. But presumably some actual semantically meaningful words would be needed too. That would be straight forward to do too though. Someone has probably already done it. The advantage would be you could get probability metrics on a broader set of text. Good data is the key thing though.

The fact that OP felt the need to add a disclaimer suggest that they expect people who write such abominations to search for detectors :)

Who knows, though. Maybe there is a marketing dude who once thought "maybe that's too much?". Naah.

Hm. I wonder how well this works versus a large LLM. Seems like something a very strong LLM should be able to handle well with the right prompting.

If you can handle it with just phrases that would save a lot of time and money though.

Yeah, presumably words with vague meanings, low semantic quality are easy to find. Still, a Bert model for it would be fun.

It counts these phrases that the author doesn't like:

https://github.com/pilotpirxie/bullshit-detector/blob/main/s...

By including this file this project should therefore correctly give itself a very high bullshit score. It's performance art really.

That is equivalent to noticing that antivirus applications would flag theme themselves as viruses if they would review their own virus samples.

In applications I write, I store all example files separately for this reason and others. Even LLMs store the training data separately.

Many duplicates

Haven't looked at the workings exactly, but this can sometimes be a deliberate choice to weight the options. A number property could work, but alas. Easier to yank and paste.

I considered that. Ai also tends to list duplicates when generating exhaustive list.

Totally fair, just spittin' in the wind

Same here lol

Love this