A couple of weeks ago I was in a YouTube video which went viral. The video is in a fun format, where Kerry from GenAI Nerds acts as host and interviews guests. Fortunately Dusty from NVIDIA was on the show to answer all the hard questions. Looky here:
I’ll point out that I don’t usually let people photograph my aura that you see in the thumbnail here. It’s very special, and requires sensitive photographic equipment to get the full hyper-spectral effect. Dusty is in another short segment:
The Interesting Bits
These are the first videos on the channel. The channel as of this writing has 1.4 million views and 331 thousand subscribers in just two weeks! To put that into perspective, the JetsonHacks YouTube channel is reaching the 35K subscriber mark in its tenth year. Wowie, and congrats to the GenAI Nerds team!
A Longer Answer
During the long form interview, one of the questions asked was “Why are LLMs bad at math?”. That’s a really interesting question that people ask. The real question is different, of course. The question is “Why would LLMs be good at math?”.
An overly simple view of a LLM (Large Language Model) is that it is very good at guessing the next word given a context. It does this using computational statistics. Imagine you have a machine which reads massive amounts of text from books, articles, websites and so on. Big LLMs have read the entire public Internet, and then some. They’ve seen it all.
The machine learns patterns in the text, and becomes really good at predicting what word comes next in a sentence based on preceding words. It doesn’t “know” what the words mean, but is really good at predicting how words fit together.
This process is much more mechanical/computational than this description. But after going through this training process, the result is the LLM models you now know and love or which scare you to death. People seem to be on one side or the other.
But Why No Math?
This is where everything gets a little philosophical. Some people argue that LLMs exhibit intelligence, it be knowing things. It distills knowledge, much more than any one person knows. On the other side of the argument, folks say that the LLM doesn’t actually “know” anything. The LLM represents a less accurate data retrieval mechanism than a large database. They also make the case that until a machine is embodied, it doesn’t have real world knowledge. By embodied, they typically mean like a robot that interacts in the physical “real” world.
The issue with math in the LLM is that there’s no representation of the knowledge of how math works. Maybe simple arithmetic with small numbers, but nothing algorithmic. Without knowledge of how arithmetic works getting the right answer by guessing what comes next doesn’t work. Trust me, every one of my math teachers told me that I could not just guess the answers. Or they graded my papers as if that were true.
As a side note, those are the same killjoys that would be fussing at me when taking a multiple choice test. They got their feathers ruffled when I would check all of the answer selections to hedge my bets.
The other issue is that there’s a whole lot of numbers. Some people have told me that there’s an infinite amount of numbers. I’m not sure I believe them. I mean, have they every sat down and made a serious attempt to count them all? I think not.
And that’s why LLMs are really interesting. Jeff Bezos says that LLMs are ‘not inventions, they’re discoveries’. He’s not wrong. There’s a lot of exploring to do.
The post Generative AI is all the rage appeared first on JetsonHacks.
