Unfortunately that’s not really how it works. Sure you could say “parameters 1, 3, and 472 in the final layer led to an 82% chance of selecting the “buy eggs” tool call” but that doesn’t give you reasoning per se. You can dig a little deeper with something like J-space inspection but that’s still a developing technique and I don’t believe it gives clear reasoning either.
however these models aren’t really able to introspect like that. It would probably take a researcher looking at the model output for each state between each token generated in the relevant areas, and they might be able to identify how likely eggs were to be inserted into the list in this particulary instance in the particular way that it happened, but not necessarily why.
“yes I stole your shopping list from someone else.”
My best guess would be that in the original training data, examples of grocery lists had eggs very commonly, so it was likely to add eggs to a grocery list. Is that what you meant by stealing?
It’s sort of like how deterministic video game physics simulations can differ on different machines or between runs. It’s something to do with the execution environment and execution order on the parallel GPU pipelines. I don’t think it is to do with floating point imprecision, but that may be a factor too. I don’t remember the details; I read an article on it a few months ago that I can try to find for you if you’d like.
AI shouldn’t be allowed to say that it doesn’t know why it did something this.
All the steps are deterministic and reproducable. It could look at what it did and be honest “yes I stole your shopping list from someone else.”
Unfortunately that’s not really how it works. Sure you could say “parameters 1, 3, and 472 in the final layer led to an 82% chance of selecting the “buy eggs” tool call” but that doesn’t give you reasoning per se. You can dig a little deeper with something like J-space inspection but that’s still a developing technique and I don’t believe it gives clear reasoning either.
Funny if it would show the LLM was overfixated on eggs because its training dataset was half eggs
“Boss we need to train the AI but the only customer willing to give us their data is the egg factory”
“So? What are you waiting for?”
Yes,
maybe,
however these models aren’t really able to introspect like that. It would probably take a researcher looking at the model output for each state between each token generated in the relevant areas, and they might be able to identify how likely eggs were to be inserted into the list in this particulary instance in the particular way that it happened, but not necessarily why.
My best guess would be that in the original training data, examples of grocery lists had eggs very commonly, so it was likely to add eggs to a grocery list. Is that what you meant by stealing?
How could a rerunnable process like an AI generating content be deterministic but not reproduceable with identical inputs?
(Good response tho)
It’s sort of like how deterministic video game physics simulations can differ on different machines or between runs. It’s something to do with the execution environment and execution order on the parallel GPU pipelines. I don’t think it is to do with floating point imprecision, but that may be a factor too. I don’t remember the details; I read an article on it a few months ago that I can try to find for you if you’d like.
Interesting, makes sense I guess. If you do have a link I wouldn’t say no!