• marmotini_enjoyer@lemmy.blahaj.zone
    link
    fedilink
    arrow-up
    2
    ·
    2 days ago

    deterministic

    Yes,

    reproducable

    maybe,

    It could look at what it did

    however these models aren’t really able to introspect like that. It would probably take a researcher looking at the model output for each state between each token generated in the relevant areas, and they might be able to identify how likely eggs were to be inserted into the list in this particulary instance in the particular way that it happened, but not necessarily why.

    “yes I stole your shopping list from someone else.”

    My best guess would be that in the original training data, examples of grocery lists had eggs very commonly, so it was likely to add eggs to a grocery list. Is that what you meant by stealing?

    • Feathercrown@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 days ago

      How could a rerunnable process like an AI generating content be deterministic but not reproduceable with identical inputs?

      (Good response tho)

      • marmotini_enjoyer@lemmy.blahaj.zone
        link
        fedilink
        arrow-up
        2
        ·
        1 day ago

        It’s sort of like how deterministic video game physics simulations can differ on different machines or between runs. It’s something to do with the execution environment and execution order on the parallel GPU pipelines. I don’t think it is to do with floating point imprecision, but that may be a factor too. I don’t remember the details; I read an article on it a few months ago that I can try to find for you if you’d like.