Replaying the seed will only get you the same result using a particular engine and a particular random number generator
And the seed and rng are all part of the LLM. If you use the same inputs you get the same output. That’s the definition of determinism.
Determinism does not care whether or not the user/programmer understands the program or not. The word you’re probably looking for is “unpredictable” or “less predictable”.
Everybody understands what you’re saying. Yes, the source of randomness in a computer is deterministic (unless you have a physical source of entropy). Every computer program is deterministic.
Nothing about the operation of LLMs relys on that determinism or views it as useful. LLMs are probabilistic in nature and require randomness (emulated or real). When people are saying it’s not deterministic they are stating that there are multiple outputs possible for a single input (excluding the seed to the RNG because it’s not necessary to use a deterministic source of randomness).
That is fundamentally different to a compiler, which is the overall point being made.
Sometimes gcc will opt to use a randomized model to guess branch probabilities, when none are available from either profiling feedback (-fprofile-arcs) or __builtin_expect.
I’m willing to bet many people are not even aware that modern compiler optimizations use prng. Not to mention incorrect usage of the term “determinism” when referring to CS concepts. This is exactly what Linus is talking about when he cited lack of understanding of compiler behavior. I do believe there are notable differences between AI and compiler, but a lot of the reasons people give against AI are reactionary and not well thought out.
I’m done. This is a pointless exercise. You’re refusing to engage on the core point and keep dancing around on semantics. None of this makes an LLM like a compiler.
And the seed and rng are all part of the LLM. If you use the same inputs you get the same output. That’s the definition of determinism.
Determinism does not care whether or not the user/programmer understands the program or not. The word you’re probably looking for is “unpredictable” or “less predictable”.
Everybody understands what you’re saying. Yes, the source of randomness in a computer is deterministic (unless you have a physical source of entropy). Every computer program is deterministic.
Nothing about the operation of LLMs relys on that determinism or views it as useful. LLMs are probabilistic in nature and require randomness (emulated or real). When people are saying it’s not deterministic they are stating that there are multiple outputs possible for a single input (excluding the seed to the RNG because it’s not necessary to use a deterministic source of randomness).
That is fundamentally different to a compiler, which is the overall point being made.
The compiler can use rng too. From the gcc docs:
More info here: https://stackoverflow.com/questions/52974259/what-are-some-examples-of-non-determinism-in-the-c-compiler
I’m willing to bet many people are not even aware that modern compiler optimizations use prng. Not to mention incorrect usage of the term “determinism” when referring to CS concepts. This is exactly what Linus is talking about when he cited lack of understanding of compiler behavior. I do believe there are notable differences between AI and compiler, but a lot of the reasons people give against AI are reactionary and not well thought out.
I’m done. This is a pointless exercise. You’re refusing to engage on the core point and keep dancing around on semantics. None of this makes an LLM like a compiler.