This argument is old and goes back to cyber security. Things like government backdoors are the same. It's true centralizing power in the hands of a "good" would theoretically be good. But the key part is the "good". I think after repeated experiences with governments and corporations, no rational person would ever assume them to be good. There are simply too many incentives to be bad.
"We can't trust the government to control nukes! Every citizen should be able to build a nuclear weapon in their backyard with zero regulatory authorization!"
I'm also at a loss. It feels like some rube goldberg machine of connections among elites, bringing up people with no apparent connection to the article other than being big and in some way connected to someone else in it.
I can put up with a little bit of dramatic name dropping in an article at the start, but it never stops. Almost every paragraph is reads like some snappy autobiography of someone trying to impress the reader. I'm not even sure what the article is about, other than name dropping.
Not specifically about the comment you replied to, but there has been an absolute Cambrian explosion of plausible sounding projects that use really complicated words that are just AI psychosis. Recently there was someone who posted a huge Lean proof on GitHub supposedly of proving P=NP. The problem is AI is really good at deluding people if that person doesn't have domain specific knowledge. AI severely makes dunning Kruger worse
They phrase it in a confusing way, with a lot of fluff. But to my understanding what happens is
1. At some part in the transformer, calculate the Jacobian (matrix gradients for that forward pass). Do this for 1000 prompts and average them.
2. For another pass take an "activation vector" (the input vector x to that part). Apply the matrix in step 1 to this vector.
3. Unembed the output to get a token. Presumably to make this have meaning, they use the reading of past tokens after embedding, instead of a random vector in the middle of the transformer.
The high level idea appears to be you're trying to find which token, if the value were changed slightly, creates the largest downstream change.
Completely stopping people from using open source models is impossible, but if it became a federal law there could be incredible pressure exerted on US companies. For example, OpenRouter might be compelled to not accept open source requests on threat of fines.
I'm just a guy who writes code and geopolitics and big business is usually above my head
That said, I have a suspicion that if the US exerts that kind of pressure on US companies, they will just pivot to using offshore contractors that are doing the things they want
Paraphrasing him, he says that open source software which tool tens or hundreds of millions to build should be classified as a different category. And the implication being they should be banned. Why?
'These tools will become dangerously powerful, which is why nobody should be allowed to have them except through buying them from specifically me' seems a lot like motivated reasoning to me.
Seems like they haven't learned from the last time they played with fire and got their models restricted to only U.S. citizens for "national security reasons".
I feel obliged to point out the disingenuousness of what this post says and the post it quotes. The most egregious parts are payment fraud and reselling - these are speculated but not actually said to be known to be happening, which you have left out.
1. Claims of payment fraud. I actually clicked the BBC article linked about payments fraud it referred to. It was an article about a criminal syndicate stealing credit cards. It mentions buying cryptocurrencies, AI API purchases are not mentioned.
2. The claims that they are reselling the chats to AI labs. The post you cite is speculating it could be, but this is unverified.
The claims of reselling is also bizzare. Arbitrary user prompts are low quality data. If I were an AI lab, wouldn't I just pay for the API proxy and get targeted output for far less?
Also calling it bot accounts is a stretch. Bots mimick human input. These are proxy accounts.
The openness of AI is currently being held up only by Chinese companies (previously Meta, but they stopped). They're not saints, but there is not even a question in the open weight/HF community that the immense mass of Chinese talent, knowledge and resources are the only thing stopping a monopoly/duopoly from forming. In a very Cathedral vs Bazaar-esque way, China severely lacks compute, but are extremely ingenious in coming up with new optimizations, architectures, etc, which they all detail in their papers.
