Then why not train an AI on the entirety of Wikipedia? I know it’s not all correct, but that should ensure most of the information is decently accurate. Would make for a great tool if it allowed to get the same info but explained in a more casual manner.
I know it’s not all correct, but that should ensure most of the information is decently accurate
The problem is that a generative AI does not generate correct content, it generates associated content. It looks at words/term/tokens that are frequently used together to generate a context, and will extrapolate on that, continuing to provide content that looks the teaching content.
The problem is that this will generate materials that LOOKS LIKE CORRECT material, but it doesn’t generate material that IS CORRECT. Thankfully for AI, those things overlap a lot, but they don’t always.
You need an absolutely insane amount of data to train LLMs. Hundreds of billions to tens of trillions of tokens. (A token isn’t the same as a word, but with numbers this massive it doesn’t even matter for the point.)
Wikipedia just doesn’t have enough data to make an LLM off of, and even if you could do it and get okay results, it’ll only know how to write text in the style of Wikipedia. While it might be able to tell you all about the how different cultures most commonly cook eggs, I doubt you’ll get any recipe out of it that makes sense.
If you were to take some base model (such as llama or gpt) and tune it in Wikipedia data, you’ll probably get a “llama in the style of Wikipedia” result, and that may be what you want, but more likely not.
Then why not train an AI on the entirety of Wikipedia? I know it’s not all correct, but that should ensure most of the information is decently accurate. Would make for a great tool if it allowed to get the same info but explained in a more casual manner.
The problem is that a generative AI does not generate correct content, it generates associated content. It looks at words/term/tokens that are frequently used together to generate a context, and will extrapolate on that, continuing to provide content that looks the teaching content.
The problem is that this will generate materials that LOOKS LIKE CORRECT material, but it doesn’t generate material that IS CORRECT. Thankfully for AI, those things overlap a lot, but they don’t always.
You need an absolutely insane amount of data to train LLMs. Hundreds of billions to tens of trillions of tokens. (A token isn’t the same as a word, but with numbers this massive it doesn’t even matter for the point.)
Wikipedia just doesn’t have enough data to make an LLM off of, and even if you could do it and get okay results, it’ll only know how to write text in the style of Wikipedia. While it might be able to tell you all about the how different cultures most commonly cook eggs, I doubt you’ll get any recipe out of it that makes sense.
If you were to take some base model (such as llama or gpt) and tune it in Wikipedia data, you’ll probably get a “llama in the style of Wikipedia” result, and that may be what you want, but more likely not.
There’s a simple English Wikipedia.