Groq, led by former Google engineer and CEO Jonathan Ross, claims to have created the first Language Processing Unit (LPU) which it says can deliver the fastest speeds for artificial intelligence applications.
It's a bold claim, but one that the latest demonstrations more than back up, suggesting it could well become an absolute game-changer for AI.
Ross, who previously designed Google's tensor processing unit (TPU), launched Groq in 2016 to create a chip capable of running deep learning inference tasks more efficiently than existing CPUs and GPUs.
Lightning fast
The company's Tensor Stream Processor (TSP) resembles an assembly line, processing data tasks in a sequential and organized manner. In contrast, a GPU is similar to a static workstation, where workers come and go to apply processing steps. The efficiency of the TSP became evident with the rise of generative AI, leading Ross to rename the TSP the Language Processing Unit (LPU) to increase its recognizability.
Unlike GPUs, LPUs use an optimized approach, eliminating the need for complex programming hardware and ensuring consistent latency and performance. LPUs are also energy efficient, reducing the overhead of managing multiple threads and preventing underutilization of cores. Groq's scalable chip design allows multiple TSPs to be linked without traditional bottlenecks, simplifying hardware requirements for large-scale AI models.
The first public demonstration of Groq was a lightning-fast AI response engine that generated responses containing hundreds of words in less than a second. Matt Shumer posted the test on X and says that more than 3/4 of the time was spent searching and not generating.
The first public demo using Groq: a lightning-fast AI response engine. Write objective, cited answers in hundreds of words in less than a second. More than 3/4 of the time is spent searching, not generating! The LLM runs in a fraction of a second. pic.twitter.com/QaDXixgSzpFebruary 19, 2024
While that's impressive, seeing Groq go toe-to-toe with Chat-GPT is something else.
If you want to try Groq for yourself and get an idea of how fast it can be for AI, head over to this chat page. Use the drop-down menu on the left to switch between the different models available.
More from TechRadar Pro
