Very nice! I've been looking for more of this kind of information.
Some additional stuff:
-A jailbreak+local builder to see exactly what function calls look like when they actually go into OPENAI's model. Note how many aspects of the JSON schema are ignored.
https://gist.github.com/CGamesPlay/dd4f108f27e2eec145eedf5c7...
Token counting is really useful for use cases where you can insert multiple tasks into a single completion call. That way you can reuse system messages and functions across many tasks as long as you stay under the context limit.
There's also something fucky going on with FC outputs (single/double quotes randomly, banned logits still showing up there) but I haven't narrowed down exactly what it is.
Some additional stuff: -A jailbreak+local builder to see exactly what function calls look like when they actually go into OPENAI's model. Note how many aspects of the JSON schema are ignored. https://gist.github.com/CGamesPlay/dd4f108f27e2eec145eedf5c7...
-A convenient tokeniser https://tiktokenizer.vercel.app/
Token counting is really useful for use cases where you can insert multiple tasks into a single completion call. That way you can reuse system messages and functions across many tasks as long as you stay under the context limit.
There's also something fucky going on with FC outputs (single/double quotes randomly, banned logits still showing up there) but I haven't narrowed down exactly what it is.