No company has benefited as much from the rise of AI as Nvidia. Nvidia, known for its work in making semiconductors that power the data centers responsible for delivering artificial intelligence, is now the most valuable company in the world, having increased its share price by 181% since the beginning of the year.
This absolute dominance is largely based on Nvidia’s position as the undisputed leader in the AI hardware space. The company’s rivals simply cannot compete with the superior performance of its graphics processing units (GPUs), which are the foundation of widespread adoption of AI tools, as Nvidia’s technology is better in terms of versatility and raw performance.
However, despite so many recent successes, there are still several challenges that threaten to undermine Nvidia's competitive advantage. The cost of producing these GPUs remains very high, while the accelerated use of AI raises environmental concerns, as the technology involved is very energy intensive.
With this in mind, is Nvidia likely to continue its meteoric rise in the coming months and years? Or will the tech giant's competitors find a way to close the gap?
Vice President of DAI Magister.
Segregating the market
The AI semiconductor market can be divided into two sections: training and inference applications, where training is only done for GPUs in data centers and inference is done on servers or edge devices. As a result, there are essentially three market segments that organizations looking to gain a foothold in the industry can target.
AI inference at the edge is driven by the need to improve data security, as reducing reliance on cloud servers minimizes the risk of data breaches. Additionally, edge devices offer real-time data processing, zero latency, and autonomy, improving overall performance.
Cost savings are another important factor, as reducing reliance on expensive cloud services for AI can lead to substantial reductions in total cost of ownership. Additionally, reducing power consumption and carbon emissions from edge devices aligns with environmental, social and governance (ESG) goals.
Nvidia's monopoly of the training market
In the GPU training sector, Nvidia’s dominance is overwhelming, with a 98% market share compared to rivals such as Google and Intel. This unprecedented level of success is unlikely to fade away anytime soon, thanks to the level of performance of Nvidia’s semiconductors and the synchronized ecosystem the company has established.
Essentially, the latest generation GPUs and comprehensive software support make Nvidia the ideal solution for many data center and high-performance computing applications. As a result, any potential rival in this sector faces insurmountable barriers to entry if it intends to challenge Nvidia.
Bridging the edge inference gap
It is the edge AI inference market where companies have the greatest opportunity to infiltrate the semiconductor industry.
Perhaps the most notable factor here is that startups are pushing the implementation of neural processing units (NPUs), a more specialized and lower-power alternative to GPUs. NPUs are designed to accelerate the processing of AI tasks, including deep learning and inference. They can process large volumes of data in parallel and quickly run complex AI algorithms using specialized on-chip memory for efficient data storage and retrieval.
While GPUs have greater processing power and versatility, NPUs are smaller, less expensive, and more energy-efficient. Contrary to popular belief, NPUs can also outperform GPUs in specific AI tasks due to their specialized architecture.
Moreover, the business model adopted by many startups favoring NPUs allows them to focus finite resources on research, development, intellectual property, and time-to-market, while leveraging capital-intensive foundries like TSMC and Samsung for actual chip manufacturing.
By focusing on core competencies in chip architecture and driving the advancement of NPUs, these fabless companies are transforming the AI semiconductor landscape and positioning themselves as key players driving the next wave of technological advancements.
Who is getting more attention?
California-based SiMa.ai has raised an impressive $270 million to date, with its platform accelerating the proliferation of high-performance, ultra-low-power machine learning inference in embedded edge applications. Similarly, Etched, another California company, provides specialized AI-based computing hardware for Transformers, designed to radically reduce LLM inference costs.
Texas-based Mythic is valued at more than $500 million and offers desktop-grade GPUs on a chip that run at a fraction of the cost without compromising performance or accuracy. Another company generating significant interest is Quadric, which develops edge processors for on-device AI computing.
Promising companies have emerged in Europe, despite the lack of a coherent AI semiconductor strategy. Axelera AI, a Netherlands-based software AI platform for accelerating machine vision on edge devices, announced a €63 million Series B funding round in late June 2024, led by EICF, Samsung Catalyst and Innovation Industries.
A cost-effective and energy-efficient alternative
The startups mentioned above have such great growth potential because they thrive in areas where Nvidia is vulnerable. NPU edge devices are a viable alternative to GPUs, addressing pressing issues of cost, size, and power consumption, and their applications span from industrial IoT to autonomous vehicles.
Toppling Nvidia's power won't be an easy task, but with larger tech giants like Microsoft, AWS, and Google actively looking to develop or acquire AI chip technologies, market consolidation is on the horizon and could alter the balance of power.
We have listed the best cloud storage for business.
This article was produced as part of TechRadarPro's Expert Insights channel, where we showcase the brightest and brightest minds in the tech industry today. The views expressed here are those of the author, and not necessarily those of TechRadarPro or Future plc. If you're interested in contributing, find out more here:
