Comment: How to minimise the cost of AI during an economic downturn

In the past few months, emerging economic trends and the government’s fiscal and monetary policy decisions have impacted the AI space. The Autumn Statement 2022 and the Economic and fiscal outlook, November 2022, lay down bleak growth projections for the next few quarters. The AI industry will feel the impact of several rapidly shifting economic variables in the coming months.

Adopting AI-based solutions can bring massive boosts to the productivity of business workflows, however excessive costs, now reverberated by declining revenues and rising energy prices, can prevent companies from adopting and scaling AI. While there is no easy price estimation, the costs of developing, testing, and deploying a basic AI system can reach up to $50,000. 

In the current economic climate, investing in a costly AI solution may have to be reconsidered. For those who have already implemented AI solutions, the costs of maintaining and updating these systems can also be a significant pain point. This is particularly true for those who may be currently facing rising costs in cloud-based deployment or energy consumption.

While these costs vary with the number of predictions made and the size of the models trained, the expenditure of maintaining an AI solution also can change with other factors that often go unnoticed. Mitigating some of these factors can make AI solutions far less expensive, especially shielding companies against cost increases that happen during times of an economic downturn. 

The unexpected Costs of bad code

Driven by the need to experiment more with data science elements of machine learning model development, those in the data science pipeline may spend less time refining code to achieve optimal code quality. Bad code adds to the costs of maintaining AI solutions, and its burden often goes unnoticed. 

A quick search will reveal that bad code wastes developer time, takes up resources, and reduces business profitability. There are many estimations of the monetary value of losses caused due to bad code. For instance, according to Stripe calculations, bad code equates to nearly $85 billion worldwide in opportunity costs lost annually. Even though the value of investing in maintaining code quality is often highlighted, the problem of sub-par code seems to exacerbate, with no definitive solutions. 

How can code-optimisation help with AI cost-saving?

AI applications are subject to change along three axes: model, code, and data. We look at two of these axes to understand how AI can be optimised for cost-efficient AI. 

Model-level Optimisation 

ML models are often at the core of AI applications. Optimising the process of producing ML-based predictions is one way to reduce the costs of implementing AI solutions.

ML models can be implemented using different frameworks/libraries, where some implementations can be more cost-efficient than others.

However, converting an ML model to a different format is complex. It requires developers with knowledge of the frameworks, who will have to conduct a manual conversion to the desired implementation.

A solution to this is utilising intermediate representations (IR) - a type of middleman. IRs extract the original format and structure of the model and convert it into the desired format, allowing developers to circumvent the manual conversion process.

Code-level Optimisation

Another unforeseen cost of deploying AI and ML comes from the execution of inefficient code. Code that is inefficient consumes more memory, energy, and time to execute, leading to a higher cost of resources. Code optimisation is an often-under-utilised solution to achieving high-quality and efficient code bases. Since optimising code can often be cumbersome, it can easily be sidestepped. 

However, there are automated code optimisation solutions that can easily identify the most expensive lines of code that affect the speed of a model during training and prediction and optimise these lines of code. This can significantly bring down the costs of executing ML code, and thereby entire AI applications. 

Recent TurinTech research found that code optimisation could improve the execution time of specific ML codebases by up to around 20%. When the optimised code was evaluated in an Azure-based cloud environment, cost savings of around 30% per hour were observed, for the virtual machine size used.

The bottom line

While economic forecasts might look bleak for the next few months, your expectations for AI adoption should not be. With a carefully developed strategy that successfully utilises cutting-edge optimisation technologies, we are certain that you will be able to progress with your AI adoption, implementation, and maintenance plans at a steady pace.

Dr Leslie Kanthan, Co-Founder and CEO of TurinTech