I assumed it was coming straight from the API because of the token-by-token gene... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		iknownthing on May 30, 2024 \| parent \| context \| favorite \| on: Show HN: ChatGPT UI for rabbit holes I assumed it was coming straight from the API because of the token-by-token generation effect but maybe you're right.

rodiger on May 30, 2024 [–]

The token-by-token responses are probably API, while the "instant" loads seem to be cached.

Probably using groq based on speed of response

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact