The rise of AI computing has led to delays in the supply of AI-enabled chips as demand has outstripped supply. Global giants Microsoft, Google and AWS are increasing production of custom silicon to reduce dependence on dominant GPU suppliers NVIDIA and AMD.
As a result, APAC companies may soon find themselves using an increasing range of chip types in cloud data centers. Which chips they choose will depend on the computing power and speed required for different application workloads, cost, and relationships with cloud providers.
Major cloud providers are investing in custom silicon chips
Compute-intensive tasks, such as training a large AI language model, require enormous amounts of computing power. As demand for AI computing has increased, super-advanced semiconductor chips from companies like NVIDIA and AMD have become very expensive and difficult to secure.
Dominant hyperscale cloud providers have responded by accelerating production of custom silicon chips in 2023 and 2024. The programs will reduce dependence on dominant vendors, so they can offer AI computing services to customers around the world and in APAC.
Google unveiled its first custom ARM-based CPUs with the launch of the Axion processor during its Cloud Next conference in April 2024. Building on custom silicon work over the past decade, the move to producing its own CPUs is designed to support a variety of general-purpose computing, including CPU-based AI training.
For Google Cloud customers in APAC, the chip is expected to enhance Google's AI capabilities within its data center and will be available to Google Cloud customers later in 2024.
microsoft
Microsoft, in the same way, has presented its first in-house custom accelerator optimized for AI and generative AI tasks, which it has called Azure Maia 100 AI Accelerator. This is joined by its own ARM-based CPU, the Cobalt 100, both formally announced at Microsoft Ignite in November 2023. The company's custom AI silicon has already been used for tasks such as running Microsoft's ChatGPT 3.5 large language model. OpenAI. The global tech giant said it expected broader deployment in Azure cloud data centers for customers starting in 2024.
AWS
AWS's investment in custom silicon chips dates back to 2009. The company has released four generations of Graviton CPU processors, which have been deployed in data centers around the world, including in APAC; The processors were designed to increase the price and performance of cloud workloads. They have been joined by two generations of Inferentia for deep learning and AI inference, and two generations of Trainium for training AI models with more than 100 billion parameters.
AWS talks about choosing silicon for APAC cloud customers
At a recent AWS Summit in Australia, Dave Brown, vice president of Computing and Networking Services at AWS, told TechRepublic that the cloud provider's reason for designing custom silicon was to give customers choice and improve the “performance of prices” of available computing.
“It's been very important to offer options,” Brown said. “Our customers can find the processors and accelerators that best fit their workload. And by producing our own custom silicon, we can offer them more computing at a lower price,” he added.
NVIDIA, AMD and Intel among AWS chip suppliers
AWS has long-standing relationships with major semiconductor chip suppliers. For example, AWS's relationship with NVIDIA, the now dominant player in AI, dates back 13 years, while Intel, which launched Gaudi accelerators for AI, has been a semiconductor supplier since the cloud provider's inception. AWS has offered AMD chips in data centers since 2018.
Custom silicon option in demand due to cost pressure
Brown said the cost optimization fever that has gripped organizations over the past two years as the global economy has slowed has seen customers move to AWS Graviton in all regions, including APAC. He said the chips have been widely adopted by the market (by more than 50,000 customers worldwide), including the hyperscaler's top 100 customers. “Larger institutions are moving to Graviton because of the performance benefits and cost savings,” he said.
SEE: Cloud cost optimization tools are not enough to control cloud spending.
South Korean and Australian companies among users
Widespread deployment of custom AWS silicon is driving APAC customers to these options.
- Leonardo.Ai: Australia-based hypergrowth imaging startup Leonardo.Ai has used Inferentia and Trainium chips in training and inference of generative AI models. Brown said they had seen a 60% reduction in inference costs and a 55% improvement in latency.
- Kakaopay values: South Korean financial institution Kakaopay Securities has been “using Graviton in a big way,” Brown said. This has allowed the banking player to achieve a 20% reduction in operating costs and a 30% improvement in performance, Brown said.
Advantages of custom silicon for enterprise cloud customers
Enterprise customers in APAC could benefit from an increasing range of computing options, whether measured by performance, cost or suitability for different cloud workloads. Custom silicon options could also help organizations meet sustainability goals.
Improved performance and latency results.
The competition offered by cloud providers, along with chip suppliers, could drive advances in chip performance, whether in the high-performance computing category for training AI models or in innovation for inference, where latency is an important consideration.
Potential for further cloud cost optimization
Optimizing cloud costs has been a major issue for enterprises, as the expansion of cloud workloads has led customers to increase costs. More hardware options give customers more options to reduce overall cloud costs because they can more carefully choose the right compute.
Ability to match compute to application workloads
A growing range of custom silicon chips within cloud services will allow enterprises to better tailor their application workloads to the specific characteristics of the underlying hardware, ensuring they can use the silicon most appropriate for their use cases. that they pursue.
Improved sustainability thanks to less energy
Sustainability is predicted to become one of the top five factors for customers hiring cloud providers by 2028. Providers are responding: For example, AWS said carbon emissions can be reduced by using Graviton4 chips, which are a 60% more efficient. Custom silicon will help improve overall cloud sustainability.
