Hey, @derrak! Thanks for the feedback! I appreciate your constructive criticism of the post.
> > LLMs by nature are non-deterministic
> This is false. LLMs are functions. All appearances otherwise are an artifact of how we use them.
I completely agree that LLMs are functions, and that they are essentially programs like any other program is. What I am trying to illustrate is that these functions are not pure. They are full of side effects. And thereby produce different results based on a large number of factors.
> This fact already suggests that determinism isn’t (entirely) what you want. Because even if you _could_ use LLMs as functions (I admit you can’t always do this with frontier models), that wouldn’t make you happy.
What do you mean here by "even if you could use LLMs as functions, that wouldn't make you happy"? What do you imply by using them as functions, and by making one happy?
> Here it should be obvious that getting the same output on the same input is not very important. Who cares if the arguments to the function are renamed? Who cares if the function is implemented differently but essentially does what the spec asks?
I would disagree here. Renaming arguments or implementing the function differently, doesn't negate the fact that I still want "pure" output from a function. I don't want to have a function that once in a while produces 2+2=5 instead of 4. I want it to always be the same: two arguments (2, 2) should always result in the same output (4).
> I argue the _only_ thing that matters is that the output satisfies the intended relationship with the input.
I agree with you here, but with one addition - it "always" satisfies the relationship with the input. As long as it always does that, that is deterministic behavior.
The definition of deterministic algorithm is as follows: "A deterministic algorithm is an algorithm that, given a particular input, will always produce the same output, with the underlying machine always passing through the same sequence of states." This is what I am essentially talking about and seeking in my interactions with AI systems.
I like your take on that. I also tend to think that it just diversifies market offers. Not everyone will want to vibecode their applications. Some people will still prefer to have someone create it for them.
I hate to see that these companies are just waltzing past the legal standards that should be put on them to scrutinize their data collection which could be found unconstitutional.
And I very much agree that US needs to tighten their privacy legislature.
I haven’t heard of them, but I love the heart behind the cause and that people seem to be passionate about it and the fact that it this is a community thing.
glaucon, I agree with you. Most of feeling of "gatekeepiness" came from being a novice at using tmux and having to get used to a new workflow. tmux is as gatekeepy as vim, or other complex cli tool is - if you have never used it in the past, you might find it to be a challenge. That was all I was trying to point out. It is an entry in a diary of a noob tmux user :D
> > LLMs by nature are non-deterministic > This is false. LLMs are functions. All appearances otherwise are an artifact of how we use them.
I completely agree that LLMs are functions, and that they are essentially programs like any other program is. What I am trying to illustrate is that these functions are not pure. They are full of side effects. And thereby produce different results based on a large number of factors.
> This fact already suggests that determinism isn’t (entirely) what you want. Because even if you _could_ use LLMs as functions (I admit you can’t always do this with frontier models), that wouldn’t make you happy.
What do you mean here by "even if you could use LLMs as functions, that wouldn't make you happy"? What do you imply by using them as functions, and by making one happy?
> Here it should be obvious that getting the same output on the same input is not very important. Who cares if the arguments to the function are renamed? Who cares if the function is implemented differently but essentially does what the spec asks?
I would disagree here. Renaming arguments or implementing the function differently, doesn't negate the fact that I still want "pure" output from a function. I don't want to have a function that once in a while produces 2+2=5 instead of 4. I want it to always be the same: two arguments (2, 2) should always result in the same output (4).
> I argue the _only_ thing that matters is that the output satisfies the intended relationship with the input.
I agree with you here, but with one addition - it "always" satisfies the relationship with the input. As long as it always does that, that is deterministic behavior.
The definition of deterministic algorithm is as follows: "A deterministic algorithm is an algorithm that, given a particular input, will always produce the same output, with the underlying machine always passing through the same sequence of states." This is what I am essentially talking about and seeking in my interactions with AI systems.