JSON is awesome, but it's troublesome that it can't represent binary data (without a separate encoding, which requires metadata, and more code on both sides of the wire).
I "discovered" a format that easily solves this, which I call QSN (quoted string notation):
It's just Rust string literals with single quotes instead of double quotes. Rust string literals have \x01 for arbitrary bytes like C or Python string literals, but without the legacy of say \v and octal, and with just \u{} rather than \u and \U.
I use in Oil to unambiguously parse and print filenames, display argv arrays, etc. All of these are arbitrary binary data.
GNU tools have about 10 different ways to do this, but QSN makes it consistent and precise.
I'm expanding it to QTSV, a variant of TSV, but if you like you could also make some JSON variant. Technically using single quotes rather than double would make it backward compatible with JSON.
Interesting solution! I usually use base64 encoding unless I'm pushing lots of data, then unfortunately it's easier to make some kind of "file upload" separate from the json if you're going through http.
My demand for that too has slowed in the past couple years, mainly because it's getting easier to do more on the client. e.g. I don't need to upload an entire .docx file, if all my backend needs is ~50kB of values queried out of one of its contained .xml files. Not saying this is a _solution_ to any of the encoding questions, only that it's reduced my immediate need.
Hm it looks like it would satisfy a lot of binary data use cases, but so would MessagePack, CBOR, and probably a few dozen other formats.
A design goal of QSN is to be human readable like JSON. You can view it at the terminal, in an editor, in an e-mail, or in the browser. IMO that fits more naturally with Unix and the web.
So I guess I should make clear that it's not only about "binary data", but it's that + a few other constraints.
I looked at it more. It has both a string type (which has \xHH escapes oddly specified as code points), and a binary type (which is specified as base64 in text format -- this is bad because base64 isn't human-readable despite being ASCII).
The 2 different types makes it unsuitable for the use cases I'm targeting QSN at, which deal only with byte strings, which may be encoded in UTF-8 (like kernel APIs).
I don't see what Ion adds over MessagePack, CBOR, or several other such formats... but that's a different discussion :)
I "discovered" a format that easily solves this, which I call QSN (quoted string notation):
http://www.oilshell.org/release/latest/doc/qsn.html
It's just Rust string literals with single quotes instead of double quotes. Rust string literals have \x01 for arbitrary bytes like C or Python string literals, but without the legacy of say \v and octal, and with just \u{} rather than \u and \U.
I use in Oil to unambiguously parse and print filenames, display argv arrays, etc. All of these are arbitrary binary data.
GNU tools have about 10 different ways to do this, but QSN makes it consistent and precise.
I'm expanding it to QTSV, a variant of TSV, but if you like you could also make some JSON variant. Technically using single quotes rather than double would make it backward compatible with JSON.