Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
kcorbitt
on Dec 11, 2023
|
parent
|
context
|
favorite
| on:
Mixtral of experts
A good way to build intuition for how much text fits in a token is by pasting a block of text into a tokenizer playground, like this one:
https://huggingface.co/spaces/Xenova/the-tokenizer-playgroun...
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: