The Real Cost of Running AI Agents (And How to Cut It by 90%)

Everyone budgets for LLM tokens. Almost nobody budgets for the machines that sit around waiting to spend them. Talk to teams running agents in production and the spend splits roughly like this:

The line that surprises people: compute costs more than the LLM. Not because agents are compute-heavy, but because the standard deployment keeps a machine warm around the clock so it can be ready for the few minutes a day the agent actually runs.

The Idle Tax

A typical agent on a t3.medium costs about $30 a month. At 100 requests a day, 30 seconds each, it works 50 minutes out of 1,440. That is 3.5% utilization by wall clock, and it is worse than it sounds: count both vCPUs and you bought 2,880 vCPU-minutes a day to use about 50.

Scale it out and the tax compounds. Ten agents is $300 a month of mostly idle silicon. Fifty is $1,500, a junior engineer's salary spent keeping sleeping processes company.

Why Stock Serverless Does Not Fix It

The reflex answer is Lambda or Cloud Run, and the reflex is wrong for agents:

• Cold starts. Loading an ML stack takes 10 to 30 seconds of imports and page faults before any work happens
• The 15-minute cap. Long agent workflows do not fit
• Statelessness. Conversation history and tool state vanish between invocations, so you rebuild them from a database every time and pay the latency
• Pricing shape. Per-millisecond billing on a workload that is mostly blocked on I/O

Cloud Run is closer than Lambda, but it still bills per second of instance uptime, and an agent that must be reachable is an instance that is up.

The Sleep/Wake Model

Maritime's approach targets the actual cost driver, which is existence, not execution:

1. Checkpoint on idle. After a configurable timeout, the agent's full state is serialized to disk and the process exits
2. Restore on demand. A trigger arrives and the checkpoint is restored, with memory and connections intact, in about a second
3. Charge a flat fee. Because a sleeping agent costs almost nothing to keep, hosting is a flat $1 a month per agent, no matter how often it runs

That is not shaving margins. It is deleting the idle tax.

What You Give Up

Honest tradeoffs, because there are some:

• Wake latency. The first request after idle pays the restore. Sub-second on our Firecracker hosts, invisible for webhooks and cron, noticeable on a live chat's first message
• Checkpoint bounds. An agent holding many gigabytes of hot in-memory state is better off never sleeping; set its idle timeout to never
• Concurrency during wake. Requests that arrive while the agent restores are queued for that second. Sustained high-concurrency workloads want an agent that never sleeps

Cutting Costs Without Switching Platforms

Even if you never touch Maritime, stop paying for waiting:

1. Right-size the box

Profile actual usage before picking an instance. Most agents idle at a few hundred megabytes and spike briefly. A t3.small at $15 handles more agent workloads than teams expect.

2. Spot instances for interruptible work

Batch and background agents that tolerate restarts run 60 to 90% cheaper on spot capacity.

3. Batch the wake-ups

If events tolerate a few minutes of delay, queue them and process in batches. Fewer wake cycles, better cache locality, lower cost.

4. Cache LLM responses

Agents answer the same questions more often than you think. A cache in front of the model trims 30 to 50% off API spend for repetitive workloads.

5. Hard token budgets

One runaway loop can spend a month of LLM budget in an afternoon. Per-request and per-day caps turn that into a bounded failure instead of an invoice.

The Bottom Line

The expensive part of running agents is not the thinking, it is the waiting. Fix the architecture so you pay for work instead of existence, and the largest line item in the budget mostly disappears.