Commercial vs Open-Source LLM: Bedrock, EC2 or Colo?

The short answer: start with a commercial model, or a managed open-weight model on Amazon Bedrock. Move to your own GPU servers on EC2 only when you need full control of the model, or your traffic is steady enough that renting GPUs costs less than paying per use. Buy servers for a colocation data centre only when you run large, round-the-clock workloads and have a team to run them.

Every company building with AI hits this question early. The answer shapes your cost, your speed and your risk for years, so it is worth ten minutes to get the basics right.

First, three words explained

LLM (large language model): the AI “brain” that reads text and writes answers. Claude, GPT, Gemini, Llama and Mistral are all LLMs.
Token: a small chunk of text, roughly three-quarters of a word. Most providers charge per million tokens sent in and per million tokens written out.
Open-source vs open-weight: most “open-source” LLMs are really open-weight. You can download the trained model and run it yourself, but the training data is usually private and each licence has its own rules. Read the licence before you put one in a product.

The four options, in plain words

1. A commercial LLM through an API

This is like buying electricity from the grid. You send text to a provider such as Anthropic, OpenAI or Google, and pay only for what you use.

Good: the best answer quality available today, nothing to install, new versions arrive on their own.
Watch out: you can’t change how the model works inside, and the provider decides when old versions retire.

One myth to drop: “commercial” does not have to mean your data leaves your cloud. Many commercial models are also sold inside cloud platforms, such as Claude on Amazon Bedrock, so requests can stay in your own cloud account and chosen region.

2. An open-weight model on Amazon Bedrock

Amazon runs popular open-weight models for you, including models from Meta, Mistral, Qwen, DeepSeek and OpenAI’s open-weight family. You still pay per token and run no servers.

Good: low prices, a wide choice of models behind one API, and no machines to manage. If you fine-tune a model yourself, Bedrock can import it for supported model types, with no import fee. For jobs that don’t need an instant answer, batch mode costs about half the normal price on selected models.
Watch out: you can use only the models and versions AWS offers, in the regions where they are offered. Check region availability before you design around one.

3. An open-weight model on your own EC2 GPU servers

Here you rent GPU machines from AWS and run the model yourself with serving software such as vLLM. You pay by the hour, whether the machine is busy or idle.

Good: any model, any version, full freedom to fine-tune, and everything stays inside your private network.
Watch out: your team now owns scaling, patching, uptime and security. An idle GPU costs the same as a busy one.

4. An open-weight model on your own servers in a colocation facility

You buy GPU servers and place them in a colocation data centre, a building that rents you space, power, cooling and internet links.

Good: the lowest cost per answer when the servers are busy almost all the time, and full physical control.
Watch out: a large upfront spend, delivery lead times for GPU servers that can run to months, and a team that knows hardware, networking and security. Servers also need replacing every few years.

The further right you go, the more of the stack you run yourself — Who manages what: four hosting options across five layers

Read the picture from left to right: each step gives you more control and more chores. Nothing on the right is “better” by default. It only pays off when you need that control.

How to decide: four questions

1. Where must the data and the model live? If a contract or regulator says the model itself must run on machines you control, rule out the API options. If the rule is only “data stays in our cloud region”, commercial models inside your cloud may still qualify.

2. How good do the answers need to be? Don’t trust public leaderboards. Take 50 to 100 real tasks from your business, run them through two or three candidate models, and score the answers. Top commercial models usually lead on hard reasoning. Strong open-weight models are often good enough for sorting, summarising and pulling fields out of text.

3. How steady is your traffic? Paying per token wins when usage is small or spiky. Rented GPUs win when they stay busy most of the day. Owned servers win only when they stay busy nearly all day, every day, for years.

4. Who will run it? Self-hosting is a product, not a one-time setup. If you don’t have people on call for GPU servers at 2 a.m., stay with managed options.

Two questions point you to one of four ways to run an LLM — Decision tree: two questions, four outcomes

Two questions settle most cases. Start at the top: if you don’t need to run the servers yourself, quality decides between a commercial API and Bedrock. If you do, your volume and your team decide between EC2 and colocation.

A simple cost check before you commit

For any self-hosted option, work out your real cost per million tokens:

Cost per million tokens=monthly server cost + monthly people costmillions of tokens served per month

Then compare it with the per-token price of the API options. Three things are easy to forget: at least two servers for uptime, the engineer hours to run them, and the hours your GPUs sit idle at night and on weekends. Busy hardware is cheap. Idle hardware is the most expensive option on the list.

The architect’s advice: don’t marry one model

Put a thin gateway between your app and the model. Then switching providers is a settings change, not a rewrite.
Route by difficulty. Send easy jobs (sorting, short summaries) to small, cheap models and hard jobs (multi-step reasoning) to the strongest model.
Keep a test set of real tasks. It lets you switch models safely whenever prices or quality change.
Start managed, then optimise. Prove the business value first. Move to self-hosting when the numbers, not the excitement, say so.

Most mature teams end up hybrid: a commercial model for the hardest work, managed open-weight models for volume, and self-hosting only where control is a hard requirement.

Common mistakes

Choosing a model from benchmark charts instead of your own data.
Comparing token prices with GPU prices without counting people and idle time.
Skipping the licence terms of an open-weight model.
Assuming self-hosting is safer by default. Security comes from access controls, encryption and monitoring, not from where the server sits.

How SHVX can help

Our Agent Engineering team runs a short model-selection assessment on your own documents and tasks. We compare quality, cost and speed across the options above and give you a written recommendation with the cost math.

Book an assessment info@shvx.in

Learn more about Agent Engineering.

FAQ

Is an open-source LLM free?

The model download is usually free. Running it is not: you pay for GPUs, power and the people who keep it running.

Is Bedrock cheaper than running my own model on EC2?

For light or uneven traffic, usually yes, because you pay only for what you use. For heavy, steady traffic, EC2 can cost less if your GPUs stay busy.

Can I use a commercial model without my data leaving my cloud?

Often yes. Several commercial models are offered inside major cloud platforms, so requests stay in your account and region. Check your provider’s data-handling terms.

When does colocation make sense?

When you run large workloads around the clock for several years, need physical control of the hardware, and have a team to operate it.