In a rapidly evolving AI landscape, Australian organisations find themselves at a critical juncture.
The potential for significant financial gains associated with AI is clear: some reports show that adopting an AI portfolio can generate more than $100 million in incremental EBITDA. But the path to achieving ROI is fraught with challenges.
Up to 85% of AI implementations in enterprises fail to deliver on their promises. AI’s high failure rate, which outstrips even the notorious difficulties of previous digital transformation efforts, underscores the risks involved.
When AI implementations fail, the impact can be catastrophic. Australia exemplifies the risks posed by AI, as demonstrated by the “robo debt” scandal, which became so damaging to Australians that a Royal Commission was convened to investigate it.
A Gartner analyst offers advice
While many are excited about the possibilities offered by AI, reports show that 80% of Australians are deeply concerned about the risks posed by AI and feel that these risks should be considered a “global priority”.
Yet despite the risks and societal hesitancy, CIOs are pouring money into AI projects: KPMG research showed that more than half of Australian companies are spending 10-20% of their budget on AI.
This only increases the pressure on the CIO and IT team to ensure AI projects prove their value. Organizations looking to make AI a long-term investment opportunity must overcome concerns about risk. Gartner research shows that estimating and proving business value is the biggest barrier to AI projects.
Nate Suda, senior director analyst for financial technology, value and risk at Gartner, told TechRepublic that the challenges many organizations face in articulating the value of AI include cost management, productivity benefits, and the strategic approaches needed to ensure AI investments translate into tangible business value.
Understanding cost dynamics
Cost management is a key hurdle in AI implementations. Unlike traditional search engines, where expenses are minimal, generative AI incurs substantial costs due to its interactive nature.
Users often engage in multiple exchanges to refine responses, which exponentially increases costs. Each interaction, measured in tokens, increases the expense. This cost can skyrocket if user behavior deviates from initial assumptions.
As Suda said, “One of the biggest cost variables is human interaction. With generative AI, it’s not enough to just type a question and get a perfect answer. It may take multiple iterations, and you’re charged for every word in the question and answer. If your cost model assumes a single interaction and users end up having multiple interactions, your costs can multiply dramatically.”
To mitigate this risk, organizations are adopting a “slow scale” strategy. Instead of a rapid, large-scale deployment, they initially roll out the planned AI deployment with a limited number of users before gradually increasing the number of users.
This iterative approach allows companies to observe the performance of ambitious AI projects and adjust them based on actual usage patterns, ensuring they can model costs more accurately and avoid financial surprises.
“The best organizations are scaling very slowly,” Suda said. “They might start with 10 users in the first month, then 20 in the second month, and so on. This method helps them understand real usage and costs in a real environment.”
The productivity enigma
While AI promises to improve productivity, translating these improvements into measurable financial benefits is complex. Suda said that simple time savings, as demonstrated by tools like Microsoft Copilot, do not inherently equate to revenue generation or cost reduction.
“It is necessary to be very clear about what productivity means and how to transform that benefit into value, whether it is income generation or cost reduction,” said Suda.
He also stressed the need to distinguish between benefits and value. Benefits such as increased speed, better customer experience and increased productivity are important, but they only become valuable when they contribute to the bottom line.
For example, generative AI might shorten the time required for a sequence of professional services, but unless this efficiency translates into increased revenue or lower costs, it becomes an example of AI failing to deliver on its promised value.
The risk of cost overruns
Another crucial point Suda pointed out is the risk of cost overruns due to unforeseen user behavior. If an AI system proves very popular and its usage exceeds expectations, the resulting costs can be astronomical. This scenario highlights the importance of meticulous planning and real-time monitoring of AI deployments to effectively manage and predict expenses.
“If users love AI and use it widely, costs can skyrocket,” Suda said. “That’s why it’s so important to understand and model user behavior.”
Strategic deployment: defend, extend, take down
Gartner has developed a three-tiered framework to explain how AI can deliver value while balancing the associated risk. Each “tier” of AI implementation, called “Defend, Extend, and Upend,” offers different potential risks and benefits.
- Defend: This involves small, incremental improvements, such as using AI to enhance existing tools. These low-cost, low-risk initiatives can yield small wins. The challenge, however, is adding up these wins to generate meaningful financial returns. According to Suda, the articulated benefits of many of these projects are marginal, making it difficult for the CIO and IT team to keep moving forward with the full support of the organization.
- Extend: In this case, AI is integrated into existing applications to deliver targeted improvements. These initiatives require careful planning and execution to ensure they deliver the intended value, but are also more likely to deliver noticeable benefits.
- Turn upside down: The most ambitious and high-risk approach involves developing new AI-based models or applications. While the potential rewards are substantial, the investment required is significant and the odds of success are lower.
AI cannot be avoided, but it must be managed effectively
As with digital transformation, trying to be too ambitious with AI from the start will likely result in cost overruns and slow ROI, leading to board and executive frustration, or even abandonment of the project.
Instead, CIOs should take a cautious and measured approach. As Suda mentioned, companies should ensure that the solutions being implemented are scalable and achieve a return on investment that can be articulated from the outset.
