Bytedance will still have access to the same open internet as OpenAI had. But it does not have the weights and the parameters.
However by asking ChatGPT questions and recording their answers, you can use that to fine-tune the model they are creating. They can tell their own model to answer the way ChatGPT answered.
In short they are copying how ChatGPT would answer. and not finding out for themselves by creating their own Question and answer (or completion) datasets.
That is what RLHF is.
But anyway, Chinese companies will be able to compete here, as this will be commoditized quickly and China is the king of creating and selling commodities.
What is clear is that Google really has no 'God model' that they were holding back all along.
Gemini Ultra is barely beating ChatGPT in their manufactured benchmarks, and this is all that they got.
What this means is that those who are saying, including people at Google, that they have better models but are not releasing them in the name of AI safety, imply that, at least in the realm of LLMs, Google DeepMind had nothing all along.
I am now convinced that Google DeepMind really had nothing in terms of state-of-the-art language models (SOTA LLMs). They were just bluffing. I remember when ChatGPT was released; Google was saying that they had much better models they were not releasing due to AI safety. Then they released Palm and Palm 2, saying it's time to beat ChatGPT with these models. However, it was not a good model.
They then hyped up Gemini, and if Gemini Ultra is the best they have, I am not convinced that they have a better model.
Sundar's code red was genuinely alarming because they had to dig deep to make this Gemini model work, and they still ended up with a fake video. Even if Gemini was legitimate, it did not beat GPT-4 by leaps and bounds, and now GPT-5 is on the horizon, putting them a year behind. It makes me question if they had a secret powerful model all along
I am now convinced that Google Deepmind really had nothing in terms of SOTA LLMs. They were just bluffing.
I remember when chatgpt was released Google was saying that they had much much better models that they are not releasing because they for AI Safety. Then theu released palm and palm 2 saying that it is time to release these models to beat ChatGPT. It was not a good model.
The they hyped up Gemini, and if Gemini Ultra is the best they have, I am not convinced that they have a better model. So this is it.
So in one year, we went from Google has to have the best model, they just do not want to release to they have the infrastructure and data and the talent to make the best model. Why they really had was nothing.
- 2k/month is low. Depending on the services you provide.
- There is an 80/20 rule on per client revenue.
So this already makes the math wrong with these two assumptions.
Actually the whole article reminded me of the Law of Diminishing Marginal Returns applied with wrong assumptions.
It would be great if I can just enter "space" in the app and it just lets me talk to it. Keyboard shortcuts!
BTW I have a lot of these ChatGPT UI apps installed, mostly free and open-source. Perhaps this is really the era of going back to just talking to a chat interface like the old times.
How about training data from interactions just by sheer usage numbers? Google does not have that.
There is a reason why the quality of ChatGPT responses are better. RLHF.I am not sure though how Google can be 3x better than OpenAI to make user switch now. They are so slow, they should be the one building the plugins.
However by asking ChatGPT questions and recording their answers, you can use that to fine-tune the model they are creating. They can tell their own model to answer the way ChatGPT answered.
In short they are copying how ChatGPT would answer. and not finding out for themselves by creating their own Question and answer (or completion) datasets.
That is what RLHF is. But anyway, Chinese companies will be able to compete here, as this will be commoditized quickly and China is the king of creating and selling commodities.