You can predict with certainty what a compilers output would be. Linus is conflating the fact that most people don’t understand how a compiler converts higher level programming language into machine code with the reality that no one can actually predict and audit what an LLM puts out.
You cannot, that’s why reproducible builds are hard. Modern compiler are made with compilers (egg/chicken problem) not assembly nor machine code and you always loose some perfect prediction.
Yes LLMs are less predictible than compiler but their output is readable to human, while machine code is very hard (yet not impossible).
Is that a bad things that 99% of devs have forgot how to write code cutting holes in paper? Tools are changing and while a direct comparison between the move from paper to COBOL or assembly to C is not the same as python to LLM generated it’s hard to deny that LLM will change how human write code.
Thanks for that explanation. Are you saying that if I rebuild the exact same code on the same machine I could get different results for a given compiler? I think this would render solutions like docker useless. Could you elaborate?
Yes you can get a different result. Compilers are designed to be deterministic, but a build is more than the compiler. Usually the bytes differ but the behavior is identical.
Docker is far from being reproducible!
It works on my machine ≠ reproducible.
But reproducible will make it works on your machine.
Using a compiler does not guarantee you to get byte-perfect output each time, especially not on different machines.
That is, of course, the whole point of docker (making sure the results of compiling are identical). I totally get the different machines giving slightly different results. This is not really in the same realm of the dramatically different output a LLM will give though.
No the goal of Docker is not to make sure the results of compiling are identitcal. It is to make it work on your machine, there is some nuance to it.
This is more the goal of something like nixpkgs (NixOS).
This is not really in the same realm of the dramatically different output a LLM will give though.
Yes this is not same realm but my point was that eve if LLM would give you different results each time the result is transparent and editable. Also developers tools evolve and Linus have more technical knowledge to use this tool in a way that will gave him satisfying result than me or a junior dev with 1.5 years of experience. He was against LLM generated code a few years ago, not because he hated on AI but because this tool was giving him unsatisfiying results. If compilers were not working 50% of the time he would probably write everything in assembly. LLM models and their coding harness cannot be compared to what they were 2 years ago. And most of the devs are using them. Will that transform bad devs into good ones? Probably not. Does that mean that LLM generated or assisted code is the way to go? Not necessarly. But it means that it’s a tool that find its use in the majority of development teams and thus that they find it quite useful.
There is some issues with it and I’m not here to deny them. Whether they are social, environmental, capitalist, monopolist, learning. Like every new disruptive tool we use them in ways we won’t in the future and I think it’s similar with AI.
He is saying that you have to understand your contribution, so whatever the LLM spits out, you need to thoroughly review and double-check that it makes sense. So, in particular, he’s not trying to rate these tools, he’s merely saying make sure the contribution is objectively good, then he doesn’t care how you got there.
My issue is him saying don’t use a compiler if you don’t really understand the machine language it creates. This conflates LLM with a compiler as a tool when they are inherently different. This is disingenuous because it is possible to understand and audit what a compiler creates. You cannot audit what an LLM creates in even close to the same way.
But why can you not audit what the LLM creates? You can’t do it for the general case, i.e. prove that for a given prompt it always spits out a valid solution, but you can look at what it actually spit out for your concrete prompt and then you can freely decide, if those code changes are good.
I am of the opinion that a thorough review often takes similarly long to actually writing it yourself, and there’s a high chance that you miss architectural aspects, if you don’t investigate yourself.
But yeah, again, I think you’re taking the comparison too literal.
He is just saying that LLMs are a tool. You need to know how to use your tool and produce objectively good results. Then he does not care what tool you use to get there.
You don’t have to do everything with the LLM. Tracing the logical path very much still happens by looking at the code in your editor.
This is also why I’m saying it isn’t much faster. Understanding the existing code and what changes are necessary is the bulk of the work when handwriting code as well.
You *can* tell the LLM to go ham and implement the change on its own and then only look at the diff afterwards, but anyone who thinks they understand the code changes then, is lying to themselves IMHO.
Can you elaborate on the last sentence a bit from a technical perspective? I’m not using code myself to the extent that I could really judge between LLM-generated code being good or just-works. I can just imagine compiler output being strictly deterministic, while LLM-code, due to the way a LLM works in principle, should be more …unstructured?
I honestly didn’t think you could even audit an LLM. It is possible, but not in the same way you would audit code. It is my understanding, because of the probabilistic nature of how the LLM operates, you can get different results even within the same model. This makes tracing something like logic next to impossible. Why the LLM gave the result is somewhat of a mystery. It is in effect a black box. This may change in the future, but for now LLM suffers from this issue.
Predicting what a compiler outputs is quiet hard, especially if you enable optimization flags or get near any undefined behavior. But I think you’re right to say that a compiler is more predictable than any LLM.
I don’t get the point why you should be unable to audit the output of an LLM? If its used as a tool like described the user should have the knowlage to audit any output
Then I can run it again, and the output will be the same.
I can run a LLM prompt.
Then I can run it again, and the output should not be the same. The unpredictability is what makes the tech great - Nobody in decision making actually understands this, but the goal is not for identical outputs. Just like me and you, don’t respond to the same question in the same way every time
I like Linus, but comparing an LLM to a compiler is fucking stupid.
Why?
You can predict with certainty what a compilers output would be. Linus is conflating the fact that most people don’t understand how a compiler converts higher level programming language into machine code with the reality that no one can actually predict and audit what an LLM puts out.
You cannot, that’s why reproducible builds are hard. Modern compiler are made with compilers (egg/chicken problem) not assembly nor machine code and you always loose some perfect prediction.
Yes LLMs are less predictible than compiler but their output is readable to human, while machine code is very hard (yet not impossible).
Is that a bad things that 99% of devs have forgot how to write code cutting holes in paper? Tools are changing and while a direct comparison between the move from paper to COBOL or assembly to C is not the same as python to LLM generated it’s hard to deny that LLM will change how human write code.
Thanks for that explanation. Are you saying that if I rebuild the exact same code on the same machine I could get different results for a given compiler? I think this would render solutions like docker useless. Could you elaborate?
Yes you can get a different result. Compilers are designed to be deterministic, but a build is more than the compiler. Usually the bytes differ but the behavior is identical.
Docker is far from being reproducible!
It works on my machine ≠ reproducible. But reproducible will make it works on your machine.
Using a compiler does not guarantee you to get byte-perfect output each time, especially not on different machines.
Yes, that is a great explanation. Thank you.
That is, of course, the whole point of docker (making sure the results of compiling are identical). I totally get the different machines giving slightly different results. This is not really in the same realm of the dramatically different output a LLM will give though.
No the goal of Docker is not to make sure the results of compiling are identitcal. It is to make it work on your machine, there is some nuance to it.
This is more the goal of something like nixpkgs (NixOS).
Yes this is not same realm but my point was that eve if LLM would give you different results each time the result is transparent and editable. Also developers tools evolve and Linus have more technical knowledge to use this tool in a way that will gave him satisfying result than me or a junior dev with 1.5 years of experience. He was against LLM generated code a few years ago, not because he hated on AI but because this tool was giving him unsatisfiying results. If compilers were not working 50% of the time he would probably write everything in assembly. LLM models and their coding harness cannot be compared to what they were 2 years ago. And most of the devs are using them. Will that transform bad devs into good ones? Probably not. Does that mean that LLM generated or assisted code is the way to go? Not necessarly. But it means that it’s a tool that find its use in the majority of development teams and thus that they find it quite useful.
There is some issues with it and I’m not here to deny them. Whether they are social, environmental, capitalist, monopolist, learning. Like every new disruptive tool we use them in ways we won’t in the future and I think it’s similar with AI.
He is saying that you have to understand your contribution, so whatever the LLM spits out, you need to thoroughly review and double-check that it makes sense. So, in particular, he’s not trying to rate these tools, he’s merely saying make sure the contribution is objectively good, then he doesn’t care how you got there.
I got that part and I agree with it.
My issue is him saying don’t use a compiler if you don’t really understand the machine language it creates. This conflates LLM with a compiler as a tool when they are inherently different. This is disingenuous because it is possible to understand and audit what a compiler creates. You cannot audit what an LLM creates in even close to the same way.
But why can you not audit what the LLM creates? You can’t do it for the general case, i.e. prove that for a given prompt it always spits out a valid solution, but you can look at what it actually spit out for your concrete prompt and then you can freely decide, if those code changes are good.
I am of the opinion that a thorough review often takes similarly long to actually writing it yourself, and there’s a high chance that you miss architectural aspects, if you don’t investigate yourself.
But yeah, again, I think you’re taking the comparison too literal.
He is just saying that LLMs are a tool. You need to know how to use your tool and produce objectively good results. Then he does not care what tool you use to get there.
Auditing typically means tracing the logical path. How do you suggest to do this with an LLM?
I am definitely taking his words literally which is why I disagree with just this one point.
You don’t have to do everything with the LLM. Tracing the logical path very much still happens by looking at the code in your editor.
This is also why I’m saying it isn’t much faster. Understanding the existing code and what changes are necessary is the bulk of the work when handwriting code as well.
You *can* tell the LLM to go ham and implement the change on its own and then only look at the diff afterwards, but anyone who thinks they understand the code changes then, is lying to themselves IMHO.
Can you elaborate on the last sentence a bit from a technical perspective? I’m not using code myself to the extent that I could really judge between LLM-generated code being good or just-works. I can just imagine compiler output being strictly deterministic, while LLM-code, due to the way a LLM works in principle, should be more …unstructured?
I honestly didn’t think you could even audit an LLM. It is possible, but not in the same way you would audit code. It is my understanding, because of the probabilistic nature of how the LLM operates, you can get different results even within the same model. This makes tracing something like logic next to impossible. Why the LLM gave the result is somewhat of a mystery. It is in effect a black box. This may change in the future, but for now LLM suffers from this issue.
Sleeping agent in LLM is a scary topic and the tech itself has mathematically no solution for this: https://arxiv.org/html/2401.05566v2
Predicting what a compiler outputs is quiet hard, especially if you enable optimization flags or get near any undefined behavior. But I think you’re right to say that a compiler is more predictable than any LLM. I don’t get the point why you should be unable to audit the output of an LLM? If its used as a tool like described the user should have the knowlage to audit any output
I can run a compiler.
Then I can run it again, and the output will be the same.
I can run a LLM prompt.
Then I can run it again, and the output should not be the same. The unpredictability is what makes the tech great - Nobody in decision making actually understands this, but the goal is not for identical outputs. Just like me and you, don’t respond to the same question in the same way every time
For local LLMs you can set the seed and generate the same output every time.
You can get the same output from the LLM if you put the appropriate guardrails and context in place. That’s what is happening in the enterprise space.
I was just pointing out Linus was conflating the two to make his point. I don’t agree with this, but everything else he said was fine.
100%, or we’d commit prompts instead of code lol
all the undefined behaviors would like to disagree with you there champ
Can you elaborate?