Build vs. Buy in AI: Your Own Model or an API?
Build vs. buy in AI is the choice between using a model through an API or hosting it yourself. The API usually wins, unless volume is huge or data is regulated.
What do build and buy mean?
With “buy” you pay per use to a provider that maintains the model, the infrastructure and the upgrades. Your cost grows with every call. With “build” you host an open model, serve it with an inference engine, and pay for GPUs, people and operations. There is also a middle option: running an open model through a hosted inference provider that charges per token and runs the infrastructure for you. Many comparisons skip it. To understand how closed and open models differ, read open vs. closed models.
Why does the API usually win on total cost?
Hosting a model has fixed costs even when nobody uses it. Tian Pan’s engineering blog (April 2026) estimates that a 70-billion-parameter model on four to eight rented H100 GPUs costs between 15,000 and 40,000 dollars a month before staff, and calculates a 34-month break-even under optimistic assumptions. The same text argues that staff, not hardware, is the dominant cost.
Volume estimates vary by source. A 2026 article from Marka places the break-even against frontier models at roughly 100 to 256 million tokens a month, and an academic analysis on arXiv (Cost-Benefit Analysis of On-Premise LLM Deployment) concludes that hosting is generally viable only from about 50 million tokens a month, or under strict data residency requirements. Most products do not reach those volumes. These are estimates, and they change with prices, but the pattern holds: below a certain consumption level, the API is cheaper. Measure input and output tokens separately, too. Output tokens usually cost more than input tokens, so a feature that writes long answers can change the math even when its total token count looks small.
When does hosting your own model make sense?
Four reasons justify the cost:
- Massive, repetitive volume. Classifying, extracting or summarizing millions of documents a day.
- Data that cannot leave your infrastructure. Health, banking or data residency requirements. Compliance then decides, whatever the cost.
- Controlled latency. An API call adds network and queuing time you do not control. That matters for interactions with strict latency targets, and much less for asynchronous work.
- Independence from a provider. Your own model does not change price or disappear without notice.
What is hybrid routing?
It is the recommendation that repeats most across the literature on the topic. Keep complex, high-value work on a large model behind an API, and send bulk work, such as classification or extraction, to a small model you host. Tian Pan’s text tells the story of a fintech team whose API spend rose from 15,000 to 60,000 dollars a month; after moving everything in-house it hit 85,000 across servers, a part-time DevOps engineer and production incidents, and only fell to 8,000 once it split which traffic went where. It is a case reported by a blog, not an independent study, so treat it as an illustration. The same text estimates that self-hosting only the bulk traffic cuts total AI spend by 40 to 70 percent for teams spending 10,000 to 50,000 dollars a month on APIs.
Note that the 34-month figure and the 40 to 70 percent savings come from a single blog and assume specific volumes, so use them to frame the conversation, not as a forecast for your product.
What costs get forgotten?
| Cost | Why it gets missed |
|---|---|
| MLOps staff | It looks like normal engineering time, but it is most of the total |
| Inference engine upgrades | Each model change can take days of work |
| Security audits (SOC 2, HIPAA) | They are paid every year, even when the model does not change |
| Queue, retry and backoff code | Your own API needs a layer that a provider already has |
| Egress traffic and storage | Small per unit, large in aggregate |
Steps to decide
- Measure your real usage for one week: how many tokens per day, and for which kinds of task.
- Split the tasks into complex and repetitive. Usually a few query types account for most of the consumption.
- Check whether any data is regulated. If it is, the question changes.
- Compare the monthly API cost with the full cost of hosting, including people.
- Start with an API. Move only the tasks where the numbers justify it.
- Repeat the calculation every quarter, because model prices fall quickly.
Example
Illustrative example: a customer support product spends 3,000 dollars a month on an API. Most of that spend comes from classifying incoming tickets. Before hosting a model, the team estimates that a server running a small model costs about 1,500 dollars a month plus the time of someone to maintain it. The answer is not automatic: if that person costs more than the savings, the API stays. If traffic grows tenfold, the math changes.
How long does the decision hold?
Not long. API prices and the quality of open models change every few months, so a comparison made today may not hold next quarter. That is why it helps to treat the decision as reviewable: record your token volume, your monthly cost and the threshold that would trigger a review, and repeat the calculation when any of those numbers changes.
Common mistakes
- Comparing only the price per token, without staff and operations.
- Hosting a large model when a small one can do the task.
- Leaving latency and data residency until the end of the project.
- Treating a blog’s figures as valid for your case.
- Deciding once and never revisiting the decision as prices change.
When NOT to host your own model
When volume is low, when the team has never run inference in production, or when the product is still testing whether the feature works. At those stages every hour of MLOps is an hour not spent on the experience design. To decide whether the feature is worth its cost, review AI feature ROI before investing in infrastructure. And if the question is which model to use for each task, small vs. large models is the next step.