Scientists at the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim can match the speed of Nvidia's A100 GPU, but with a smaller size and significantly lower power consumption. The chip was developed using Samsung's 28-nanometer manufacturing process, a technology considered relatively old in the changing world of semiconductors.
The team, led by Professor Yoo Hoi-jun at KAIST's in-memory processing research center, has developed what it says is the world's first 'Complementary Transformer' (Transformer C) AI chip. This neuromorphic computing system mimics the structure and functioning of the human brain, using a deep learning model often used in visual data processing.
“Neuromorphic computing is a technology that not even companies like IBM and Intel have been able to implement, and we are proud to be the first in the world to run LLM with a low-power neuromorphic accelerator,” Yoo said.
Questions remain
This technology learns context and meaning by tracking relationships within data, such as words in a sentence, which is a key technology for generative AI services like ChatGPT.
During a demonstration at the ICT Ministry headquarters, team member Kim Sang-yeob showed off the chip's capabilities. On a laptop equipped with the chip, he performed tasks such as question-and-answer sessions, sentence summarization, and translations using OpenAI's LLM, GPT-2. Tasks completed at least three times faster and, in some cases, up to nine times faster than when running GPT-2 on an Internet-connected laptop.
Implementing LLM in generative AI tasks typically requires numerous GPUs and 250 watts of power, but the team claims its semiconductor uses only 1/625 the power of Nvidia's GPU for the same tasks. Plus, because it's also 1/41 the size, measuring just 4.5mm by 4.5mm, it could ultimately be used in devices like mobile phones.
However, it remains to be seen whether the chip will be able to deliver on its promises in real-world applications. As Tom Hardware reports: “While we're told the KAIST C-Transformer chip can perform the same LLM processing tasks as one of Nvidia's beefy A100 GPUs, none of the press or conference materials we've provided provide any performance metrics. direct comparison. This is a significant statistic, conspicuous by its absence, and cynics would probably assume that a performance comparison does the C-Transformer a disservice.“
