I'd frankly far prefer people not to call isatty(), and require switches for machine-readable output. The kind of tools that "barf ANSI cruft" are usually the tools I want to do so even when I'm piping their output into something else. It's a real nuisance to e.g. deal with options to preserve colours through a pipe for every element. There might well be exceptions where I'd prefer to default to checking isatty(), but they are few.
If there was any remotely standard way of signaling the preferred default, and a remotely standard command line option to toggle that default, it'd make things a lot nicer, because I totally understand where you're coming from; I get equally annoyed just in the opposite scenarios, so the situation is a nuisance for both of us.
I'd love to have a standard command-line tool for "run the following program with its output set to a pty, and then pipe the output to stdout". That's the most reliable way to get programs to display color and similar.
Any tool whose output is ever parsed, including "parsing" as simple as "tool | grep xyz", shouldn't emit terminal escapes to a pipe by default. If `xyz` has an embedded color sequence in it, that grep will fail. Or worse, produce unexpected results. (The standard color sequences end in `m`; a grep for 'msomething' could match 'something' preceded by a color sequence.)
I agree, of sorts. The problem being that we have developed a culture of making user-facing user interface and APIs the same in Unix-y tools. I think "everyone" recognises the problem, but we may have different preferences about what the default should be. Personally, I'd prefer a default way of indicating "I want API behaviour now" but defaulting to more human-friendly output, because even in most cases like "tool | grep xyz" from the command line I'm still more likely to want colours etc. to stay intact through the pipeline. Most of the time that's fine, but you're of course right it will give unexpected results some time.
I think my ideal expected behaviour (which would still not be perfect) would be something like:
* Tools defaulting to unescaped output in scripts, but with a standard short option and/or ENV var to trigger colour output.
* Tools default to colour/escaped output when run in an interactive shell even if in a pipe.
* Tools being escape sequence aware, maybe with switches to turn that behaviour off if you genuinely e.g. do want to grep for sequences that may include escapes and you want them considered.
But I'm not sure there is a good solution to this other than decoupling UI and API and having different defaults for tools that are expected to be "user facing" vs treated as API. I have an "ls" replacement on my system, for example, which changes formatting and adds more colour to my ls output, and it's obviously not named ls because the amount of stuff that breaks if "ls" isn't reliably the same as always is significant. It's still not great to have to separate this given that part of the ease of composing pipelines etc. is familiarity with the output, but maybe if coupling that with reasonably standard switches to turn on/off machine-friendly output.
I can imagine that a world like that could work (with some care to make it less error-prone for common command-line scripting, since command lines can sometimes be small scripts and evolve into large scripts). But I think it's too late to do that for CLI tools without causing widespread breakage.
I think we might be able to do better in something that isn't a traditional shell, and that uses ptys instead of pipes, together with builtins that replace standard UNIX tools with escape-aware tools.
Yeah, I think it at least means abandoning any idea of changing the API of any of the standard tools. Whether you silently replace them in a custom shell or provide alternative named replacement.
I'm halfway tempted to replace my shell with one that is more integrated with my terminal and do something like the last bit you suggested, given it can be very trivial[1] if you explicitly make the choice that for any scripting you'll use a "regular" shell.
[1] there is, in fact, a tiny single-file Ruby shell that I might be tempted to extend.
You could write a wrapper around something like https://github.com/cvolny/faketty, which can manipulate isatty() calls when used with LD_PRELOAD on Linux (and presumably DYLD_INSERT_LIBRARIES on macOS except on Apple's binaries).
If there was any remotely standard way of signaling the preferred default, and a remotely standard command line option to toggle that default, it'd make things a lot nicer, because I totally understand where you're coming from; I get equally annoyed just in the opposite scenarios, so the situation is a nuisance for both of us.