Yes, and the silicon to handle that math already exists. It’s the “GPUs” being pumped out by nVidia. Those are basically now highly specialized matrix math machines.
NPUs are low-power hardware devices for running very small AI models.
So-called "GPU"s are the devices used for similar things for the very large models that are the “hotness” right now.
A general purpose GPU still contains silicon for display, and video en/decoding
No it doesn’t. The modern datacenter-style "GPU"s that nVidia has been making and selling for use in AI datacenters have stripped out all the video related elements. They’re purpose built for doing only AI work these days. They can’t even do standard matrix multiplication work anymore because they’re so narrowly designed for AI models where they use a specialized form of lossy matrix multiplication.
The name has stuck around, but the things nVidia has been building and selling to the AI companies no longer has anything to do with graphics.
Yes, and the silicon to handle that math already exists. It’s the “GPUs” being pumped out by nVidia. Those are basically now highly specialized matrix math machines.
Then why do NPUs exist?
What I’m talking about exists on the smaller scale. Just a matter of time before its done on a larger.
A general purpose GPU still contains silicon for display, and video en/decoding. Those don’t contribute in any meainiful way.
Not to mention it doesn’t cover all of matrix math for LLM needs…
NPUs are low-power hardware devices for running very small AI models.
So-called "GPU"s are the devices used for similar things for the very large models that are the “hotness” right now.
No it doesn’t. The modern datacenter-style "GPU"s that nVidia has been making and selling for use in AI datacenters have stripped out all the video related elements. They’re purpose built for doing only AI work these days. They can’t even do standard matrix multiplication work anymore because they’re so narrowly designed for AI models where they use a specialized form of lossy matrix multiplication.
The name has stuck around, but the things nVidia has been building and selling to the AI companies no longer has anything to do with graphics.