AI Feature ROI: How to Measure Whether a Feature Is Worth Its Cost
The ROI of an AI feature is measured by outcome, not usage: value generated over total cost, tracked as cost per completed task against a pre-launch baseline.
Why measure per outcome rather than per use?
Usage is easy to count and misleading. A feature can have thousands of opens and still save no time and sell nothing. What matters is whether each outcome is worth more than it cost to produce.
With AI this matters more because cost is variable. Every model call consumes tokens (see cost per token), so the bill grows with activity, not with value. Opslyft suggests you stop looking at aggregate monthly spend and track four numbers per outcome:
| Metric | What it answers | Who uses it |
|---|---|---|
| Cost per inference | What one model call costs, including retries and infrastructure | Engineering |
| Cost per feature | Whether the feature pays for its AI spend | Product |
| Cost per customer | Which accounts are margin-negative | Business |
| AI gross margin | Revenue minus fully loaded AI cost | Finance and leadership |
A high bill can be healthy if it powers a feature that retains a lot of revenue. A low bill can be a loss if almost nobody uses the feature.
How do you build the baseline?
Before launch, record three things for the work the feature will cover: time per task, error rate and cost per task. Elastic calls this the human baseline. Without it, any improvement you claim later is guesswork. It is also the step teams skip most often, because few teams measure a process before deciding to change it.
If the process has no data, start with sampling: observe twenty or thirty real cases, note time and errors, and use them as the reference. It is rough, but it gives you something to compare against.
How do you calculate value?
Elastic proposes the agent value multiple (AVM):
Value = (cost savings + incremental revenue + margin improvement) ÷ total cost
Each term needs a data source:
- Cost savings: minutes saved per task, multiplied by the number of tasks, converted to money at the team’s real hourly cost. Also count avoided cost, such as a hire you no longer need.
- Incremental revenue: for a recommendations feature, for example, the difference in conversion and average order value between users who saw the recommendation and a control group.
- Margin improvement: how much the cost of serving each customer drops.
The control group matters as much as the formula. Without a group that doesn’t use the feature, you cannot separate the effect of AI from the normal trend of the business.
Alongside the multiple, Elastic recommends tracking two signals that explain why it moves. Task success rate: the percentage of workflows resolved without human intervention. And user acceptance: whether people actually use the solution or go back to the manual method. A feature with a good multiple on paper and low acceptance won’t keep its savings.
Which costs get forgotten?
Elastic separates three groups of cost, taking the hidden ones from Gartner:
| Type | Examples |
|---|---|
| Upfront | Licenses, pilot development, data cleaning |
| Ongoing | Token consumption, monitoring |
| Hidden | Application development and maintenance, integration with existing systems, security and governance, data management, change management and training |
To compare simple and complex tasks fairly, Elastic uses agent cost per completed task (ACCT): total spend divided by the tasks finished successfully. If the agent fails and a person redoes the work, that cost counts too.
Opslyft adds three useful details:
- Allocating shared spend. Many customers share the same model, so cost per customer depends on how you allocate it. Opslyft recommends accepting an allocation of roughly 70% of spend now rather than waiting for perfect tagging.
- Input and output don’t cost the same. According to Opslyft, output tokens cost 4 to 5 times more than input tokens, yet many teams budget with a flat one-to-one ratio. That understates features that generate a lot of text.
- Cheaper tokens don’t mean a smaller bill. When tokens get cheaper, usage grows: more users, agents that chain several calls, and longer context windows.
What steps should you follow to calculate a feature’s ROI?
- Define the task the feature covers and its baseline for time, error and cost.
- Record every cost, including the hidden ones, and assign it to the feature.
- Measure cost per completed task, not per call.
- Measure value against the control group or the baseline.
- Calculate the value multiple and review it monthly, because cost changes with usage.
- Decide: scale, adjust or retire the feature if the multiple doesn’t stay above 1.
When shouldn’t you force an ROI number?
When the feature delivers value that can’t be measured well, such as a customer’s trust in the brand, and the team tries to invent a figure. Also when volume is so low that any calculation is noise. In those cases, measure qualitative signals through interviews and a list of criteria before investing more. To decide which features enter the strategy, see when to use AI in a product; to track usage and quality, see product metrics for AI features.
Example
Illustrative example (figures invented to show the calculation): a support team handles 10,000 tickets a month and tests a feature that drafts the reply.
- Baseline: each reply takes 6 minutes. That is 60,000 minutes, about 1,000 hours a month.
- With AI: drafting takes 2 minutes, and 15% of tickets need 4 more minutes of review. The average is 2.6 minutes: about 433 hours a month.
- Savings: about 567 hours. At 25 USD per hour, that is 14,175 USD a month.
- Total cost: 200 USD in tokens, 1,500 USD for monitoring and maintenance, and 3,000 USD a month for the integration spread over time. Total: 4,700 USD.
- Multiple: 14,175 ÷ 4,700 ≈ 3.0.
Tokens are the smallest part of the cost. Now check the sensitivity: if review rises to 60% of tickets, the average goes to 4.4 minutes, savings drop to about 267 hours (6,675 USD) and the multiple falls to 1.4. The review rate decides the ROI more than the model’s price. Before scaling, the team should tune the prompt or limit the feature to ticket categories where review is low.
Common mistakes
- Launching without a baseline and claiming improvements afterward.
- Measuring opens or clicks instead of tasks resolved.
- Counting only tokens and forgetting integration, security and training.
- Budgeting with a one-to-one ratio between input and output tokens for features that generate long text.
- Having no control group and crediting AI with an improvement that was already happening.
- Ignoring the human review work when calculating cost per task.
- Reviewing ROI once a year, when cost changes every month with usage.