Written by Alex Honchar, CTO and co-founder of Neurons Lab
Hello, everyone. This article is a written version of a presentation I gave recently covering the growing topic of return on investment (ROI) for generative AI technology. If you prefer a narrative walkthrough, here is the YouTube version:
In this article, I will review several different resources to build a map of the current conversation and media coverage. Then, I’ll run some specific formulas to build a realistic case around ROI, the required amount of work, timelines, and how to measure it.
Introduction – the expectations for ROI
Let’s start with an example of the type of reports that many of us are reading about GenAI – around adoption, productivity, and pioneering.
Many reports, including this one from PwC, are covering how a majority of businesses already adopting GenAI are seeing an improvement in the quality of the company’s products and services:
Looking at the gains, for example, the potential increase in operating margin, they are typically large – up to almost 20 percent in some industries.
AI adoption can mean different things. This article from NfX outlines a typical 5-Level AI Spectrum:
Here at Neurons Lab, we see a similar picture – mainly, companies coming to us seeking support with their AI implementation are typically at the first three levels. Companies that already use AI as their main product or are AI-first have strong in-house teams.
So, the Level 1 companies tend to look for consulting, while those at Levels 2-3 typically need engineering support.
Seeking quick wins
I also recommend reading this Buyer Behavior Report from G2, which strongly reflects our conversations with clients. Many companies expect quick wins within a couple of months.
Other concerns among the C-suite, but also managers and individual contributors, tend to include security and privacy plus of course the ROI of AI. In short – how to measure it, what the variables are and how do you guarantee it?
If you look at many popular publications – for example, this article from Forbes – the answer can be very broad.
Such articles may recommend some AI projects that could provide a faster ROI. For example, this one mentions AI-enabled chatbots that can improve user satisfaction and decrease costs.
It also covers some AI projects with a longer time to ROI – but apart from recognition systems, decision support and so on, often there’s nothing very specific. In short, when I read this type of article, I expect to see numbers or specific calculations, but often I don’t.
Is there the risk of a revenue gap?
Some commentators, including venture capitalists, are raising the question of a ROI gap – Sequoia calls this AI’s $600bn Question. In this article, David Cahn continues on from his previous, also interesting article, AI’s $200bn Question.
He reaches this figure via a calculation incorporating Nvidia’s data center costs and other CapEx considerations, earnings from GPU revenue, and margin required by the end users – businesses. It also takes into account growing GPU stockpiles and major players’ market share.
His point is that in order to return the investment that companies have already made on AI – primarily in cloud computing – there is $600bn in revenue to recoup first. The article covers many reasons why this may be the case, so I recommend reading it in depth.
Cahn concludes:
- A huge amount of economic value will be created by AI, with those focused on delivering value to end users set to be rewarded.
- But don’t believe that everyone investing in AI will get rich quick.
- In reality, the road to ROI is longer, with ups and downs – but almost certainly it will be worthwhile.
So, there is an important question to answer – how do you mitigate a potential revenue gap and ensure ROI?
A simple formula for calculating the ROI of AI investments
This question isn’t only relevant for GenAI, nor is it a new one. Back in 2020, I gave a speech at a conference covering the mathematics of AI business.
My formula for ROI was quite straightforward – inspired by this great article from Eljas Linna on Medium covering the Return on Investment for Machine Learning. For a more detailed explanation of the formula you’re about to see, I recommend reading through the full article.
Your return comes from the difference between value that you expect to get – essentially, your savings in time, money, volume, whatever you want from AI – minus any mistakes that your system is going to make.
returns = value – (1 – accuracy) * cost of a mistake
In short, your return is the difference between the true value of AI that can be calculated and the cost of mistakes or inaccuracy. From this formula, you can get a calculated break-even point for your AI model, depending on its accuracy or what the cost of a mistake is in your business and the value you expect.
We should all understand that AI doesn’t fully replace manual work. It can make mistakes – it depends on the level of accuracy built into the solution.
One certainty of any AI solution should be the Human in the Loop (HITL). There is a more elaborate formula that takes the added confidence from taking a HITL into account.
These are the basics for the calculations:
- Value: Your savings in minutes to read documents. In this scenario, something that would typically take 5 minutes manually now only needs 1s.
- Cost of mistake: Let’s say if you need to fix something, it takes 20 minutes. These minutes are easily translated into the hourly rate of your personnel.
- Accuracy: And let’s say the typical accuracy of the model is 80% in this case, with the potential to increase over time.
So, as an intermediate recommendation for calculating ROI, I highly recommend reading the full article and plugging in your own values into the formula.
Based on this, you can calculate a break-even point. If you need some help, of course, reach out to us – we can help you with that.
But what this formula doesn’t really take into account are any additional costs.
Taking additional costs into account
For example, you need to hire an AI team or outsource the project. You need to pay for the cloud and computational resources.
In a nutshell, these elements are part of the aforementioned $600bn question but are not typically mentioned in these kinds of formulas.
Again, it’s not only about the ROI of one single prediction. The formula in the previous section is nice to have because it provides unit economics for your AI use case, but it doesn’t take into account your other CapEx and OpEx.
This leads me onto another article I recommend reading. It’s one about some of the work we do in Neurons Lab – one of our very popular use cases is Intelligent Document Processing (IDP) for enterprises.
Here we demonstrate a use case with an IDP system that also takes into account a HITL, because we don’t want to fully automate the process, but augment it with AI.
Here we use a very similar formula to the above one – it includes how many invoices you process, the time needed, manual costs, how much IDP the solution can handle, the accuracy rate, and savings:
 |
Read the full article as we continue these calculations over several sections. Ultimately, in 3 years we calculate a ROI of 2.6x – a good result:
 |
So, how long will it take to see ROI?
Let’s go one step further because again, plugging in numbers to the formula is one thing… But what is the geometry of this ROI? It’s not just one jump to the return, it’s going to have some curve.
Here is a curve from a project we completed in Asia, an interesting market in this case because the labor cost is not high overall, so all types of automation can be tough to justify. AI can be less beneficial than manual work and this is what we encountered.
That’s why the right calculation is really crucial. Here the curve will look something like this.
In this scenario:
- The confidence level gradually increases over 6 months from 0% to 90%
- The number of customers increases from 2 to 44, over a period of 15 months, although this only starts to happen once the solution is in place
- The solution becomes profitable after the first full year since development started
First you need to invest. When you invest in building the solution or even integrating an existing one, you pay the team, cloud costs, data costs, computational costs, etc.
In this case, it’s an investment of over $150k, before development starts. And then you expect to have some ROI, which you can see here starts to grow about 12 months after development begins.
Again, it’s about plugging in the different values for the formula – volume of documents, accuracy of the models…
You can improve the accuracy of the model, but then you need to hire a data science team to collect and label data, then retrain and test the models. As you remember, in our formula, confidence and accuracy are important pieces.
By improving the accuracy and using a high volume of documents as inputs, we can get a higher ROI.
Wrapping up
In the last example – with more detail in the full article – we have covered realistic scenarios, where the value is acquired over a year or longer. If you are not ready to automate or augment a lot of work, you will likely reach your break-even point and desired ROI much later.
We also make several points around how what will be important in such a project is your ability or capability to invest into improving the models. Again, accuracy and volume are key variables.
This curve for ROI is something you would expect from more traditional software or digital transformation projects – if you want to make it sustainable, this is probably how it’s going to look.
I hope this walkthrough will help you understand what to expect from ROI in GenAI projects. Please get in touch and let us know what questions you would like answered.
Do you want to dive deeper into unit economics formulas, the ROI curves, more specific use cases? Whatever it is, just let us know!
About Neurons Lab
Neurons Lab is an AI consultancy that provides end-to-end services – from identifying high-impact AI applications to integrating and scaling the technology. We empower companies to capitalize on AI’s capabilities.
As an AWS Advanced Partner, our global team comprises data scientists, subject matter experts, and cloud specialists supported by an extensive talent pool of 500 experts. This expertise allows us to solve the most complex AI challenges, mobilizing and delivering with outstanding speed while supporting both urgent priorities and strategic long-term needs.
Ready to leverage AI for your business? Get in touch with the team here.
