• bstix@feddit.dk
    link
    fedilink
    arrow-up
    1
    arrow-down
    1
    ·
    2 days ago

    AI shouldn’t be allowed to say that it doesn’t know why it did something this.

    All the steps are deterministic and reproducable. It could look at what it did and be honest “yes I stole your shopping list from someone else.”

    • Feathercrown@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      2 days ago

      Unfortunately that’s not really how it works. Sure you could say “parameters 1, 3, and 472 in the final layer led to an 82% chance of selecting the “buy eggs” tool call” but that doesn’t give you reasoning per se. You can dig a little deeper with something like J-space inspection but that’s still a developing technique and I don’t believe it gives clear reasoning either.

        • Feathercrown@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          “Boss we need to train the AI but the only customer willing to give us their data is the egg factory”

          “So? What are you waiting for?”

    • marmotini_enjoyer@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      2
      ·
      2 days ago

      deterministic

      Yes,

      reproducable

      maybe,

      It could look at what it did

      however these models aren’t really able to introspect like that. It would probably take a researcher looking at the model output for each state between each token generated in the relevant areas, and they might be able to identify how likely eggs were to be inserted into the list in this particulary instance in the particular way that it happened, but not necessarily why.

      “yes I stole your shopping list from someone else.”

      My best guess would be that in the original training data, examples of grocery lists had eggs very commonly, so it was likely to add eggs to a grocery list. Is that what you meant by stealing?

      • Feathercrown@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        2 days ago

        How could a rerunnable process like an AI generating content be deterministic but not reproduceable with identical inputs?

        (Good response tho)

        • marmotini_enjoyer@lemmy.blahaj.zone
          link
          fedilink
          arrow-up
          2
          ·
          1 day ago

          It’s sort of like how deterministic video game physics simulations can differ on different machines or between runs. It’s something to do with the execution environment and execution order on the parallel GPU pipelines. I don’t think it is to do with floating point imprecision, but that may be a factor too. I don’t remember the details; I read an article on it a few months ago that I can try to find for you if you’d like.