Skip to main content
Connic
Back to BlogTutorial

Hidden Costs of Self-Hosting AI Agents

We'll just deploy it on Kubernetes. Famous last words. The true cost of self-hosting AI agents versus a managed platform.

December 18, 2025(last updated: September 8, 2026)7 min readAuthor: Connic Research Team

"We'll just deploy it on Kubernetes." Teams discussing AI agent infrastructure often hear this proposal. It sounds reasonable. An existing K8s setup makes one more container seem easy.

Many teams discover six months in that it is harder than expected. The cost model below covers self-hosting AI agents in the DACH market with real numbers.

The Visible Costs (What Everyone Budgets For)

These are the costs teams typically account for when planning AI agent infrastructure:

Direct Infrastructure Costs
Cloud Run / ECS / GKE compute€460 – €1,800/mo
Message queues (SQS, Pub/Sub, Kafka)€90 – €460/mo
Database for state management€90 – €370/mo
Secrets management (Vault, AWS Secrets)€45 – €185/mo
Total Infrastructure€685 – €2,815/mo

The visible bill looks manageable. The hidden costs change the calculation.

The Hidden Costs That Kill Budgets

1. Engineering Time to Build It

Deploying the container is only the start. The team also needs to build:

  • •Webhook ingestion layer: Accept external events, validate signatures, handle retries
  • •Queue consumers: Process messages reliably with error handling and dead-letter queues
  • •Agent orchestration: Manage concurrent runs, timeouts, and state
  • •Deployment pipelines: CI/CD for agent code with versioning and rollback
  • •Observability stack: Logging, tracing, metrics, dashboards

Conservative estimate: 2–3 months of senior engineer time. Senior software engineers in Germany earn a median base salary in the €78k–€95k range per Levels.fyi Germany compensation data; with typical employer overhead (~30% on top of gross), fully loaded cost is €100k–€125k/year, or roughly €17k–€31k just to reach "it works."

2. Ongoing DevOps Burden

Infrastructure doesn't run itself. Someone needs to:

  • →Monitor for outages and performance degradation
  • →Apply security patches and updates
  • →Scale infrastructure as usage grows
  • →Debug production issues at 3am
  • →Handle on-call rotations

This usually takes at least 20–30% of a DevOps engineer's time, which works out to €1,800–€3,100/month in ongoing labour cost at fully-loaded DACH DevOps engineer rates, often from the team's most expensive engineers.

3. Tools and Software

Observability platform (Datadog, New Relic)€185 – €925/mo
Error tracking (Sentry)€45 – €275/mo
Log management (if not in observability)€90 – €460/mo
CI/CD tooling€90 – €275/mo
Total Tools€410 – €1,935/mo

4. The Opportunity Cost

The hidden cost does not show up on any invoice: what could the engineers have built instead?

Every hour spent debugging Kubernetes networking is an hour not spent on product features that differentiate the business. Every sprint on "agent infrastructure improvements" is a sprint competitors can use to ship customer-facing features.

The 12-Month TCO Calculation

Adding it up for a realistic DACH-market scenario: a mid-size team running a moderate workload of 5,000 agent runs per month, average 30-second run duration, with cited DACH salary and EU cloud pricing assumptions.

Self-Hosted (12 months)
Initial build (2.5 months senior eng. @ €110k/yr loaded)€22,900
Infrastructure (€1,300/mo × 12)€15,600
DevOps time (25% × €110k/yr)€27,500
Tools (€700/mo × 12)€8,400
Training & upskilling€4,500
Incident response buffer (1 senior week)€2,100
Compliance & security review (4 days)€4,700
Total Year 1~€85,700
Managed Platform: Connic Pro (12 months)
Initial setup (1–2 days senior eng.)€1,800
Pro subscription (€200/mo × 12), paid as credit€2,400
Additional prepaid credit (5k LLM-agent runs at 30 seconds each) = €31.50/mo€378
DevOps time€0
Observability & tooling (included)€0
Total Year 1€4,578
12-Month Savings: ~€81,100
That's enough to hire most of a junior engineer in Germany, or fund 2–3 quarters of product development that differentiates the business.

How our pricing works in this model: the monthly subscription converts into an equal amount of Project credit. Usage draws down that credit at the published per-unit rates (€0.0337 per LLM-agent start, €0.00042 per runtime second, €0.00342/GB-hour of storage, and €0.0276/synced retrieval item). This example uses a one-month term and estimates 5,000 runs per month at base rates, with no term or volume discount. Volume discounts start at the 10,000-run package. At 5,000 LLM-agent runs/month averaging 30 seconds, run starts and runtime use €231.50/month: the €200 Pro credit plus €31.50 of purchased credit. This example excludes model tokens, storage, and retrieval usage. Extra credit can be added manually or through capped auto-refill. Enterprise contracts define custom commercial terms.

Why Existing Kubernetes Is Not Enough

The most common objection deserves a direct answer.

Kubernetes provides orchestration. AI agents still have unique requirements:

  • •Unpredictable execution times: A simple query might take 2 seconds, a complex one might take 5 minutes
  • •Token tracking: Cost control requires the exact token consumption of every run
  • •Execution traces: Standard APM doesn't capture LLM reasoning steps or tool calls
  • •Hot-reload development: Existing CI/CD is rarely built for 2-second iteration cycles

The result is a custom platform on top of the existing infrastructure, bringing the calculation back to the cost estimates above.

When Self-Hosting Does Make Sense

To be fair, there are legitimate reasons to self-host:

  • 1.Extreme data sensitivity: Regulated industries where data cannot leave the organization's infrastructure (though even here, private cloud options exist)
  • 2.Massive scale: At millions of agent invocations per day, the math might favor self-hosting
  • 3.Core competency: Building expertise makes sense when AI infrastructure is the product

For teams where AI agents are a feature rather than the product, the math strongly favors managed platforms.

Validate Before Investing in Self-Hosting

Our recommendation: start with a managed platform and validate the use case before committing to infrastructure investment.

With our Basic plan, teams can:

  • •Deploy agents in minutes, not months
  • •Validate that the integration patterns work
  • •Get real usage data to inform build-vs-buy decisions
  • •Ship AI features while evaluating long-term options

If usage eventually justifies self-hosting, the required platform scope will be clear. Otherwise, as is true for most teams, the managed start avoids a very expensive learning experience.

Further detail is available in our guide to replacing self-hosted AI agents with a managed platform, our managed vs. self-hosted TCO comparison, our pricing page for the full cost comparison, and the quickstart guide.

Frequently Asked Questions

For a DACH-market team, direct infrastructure runs €685–€2,815/month. The full year-one picture adds 2–3 months of senior engineer build time (€17k–€31k at fully-loaded DACH compensation per Levels.fyi Germany), ongoing DevOps burden (€1,800–€3,100/month), and observability tooling (€410–€1,935/month). Year-one TCO for a moderate workload of 5,000 LLM-agent runs per month is around €85,000, versus around €4,600 on a managed platform like Connic Pro.

The largest hidden costs are engineering time and opportunity cost. Building a production-ready agent hosting layer (webhook ingestion, queue consumers, observability, CI/CD, and state management) takes 2–3 months of senior engineer time before the first agent runs in production. Every hour spent on infrastructure is an hour not spent on product features.

Self-hosting is worth considering in three scenarios: data cannot leave the organization's infrastructure due to regulatory requirements; the workload reaches millions of agent invocations daily and the break-even has been modeled; or AI infrastructure is the core product and building expertise creates direct competitive advantage. For most teams where agents are a feature rather than the product, managed platforms win on total cost.

At minimum: compute (Cloud Run, ECS, or GKE), message queues for event ingestion (SQS, Kafka), a database for state management, secrets management, an observability stack, CI/CD tooling, and a webhook ingestion layer with signature validation and retry logic. Each layer adds cost and ongoing engineering maintenance burden.

AI agents have unique operational requirements: execution times are unpredictable (seconds to minutes per run), per-run token and cost tracking exceeds what standard APM tools provide, execution traces must capture LLM reasoning steps and tool calls, and the platform needs to support hot-reload iteration cycles. These differences require a custom platform layer on top of the existing infrastructure.

More from the Blog

Tutorial

Expose the Invoice Processor Template as an MCP Tool

Deploy Connic's invoice template as an MCP tool, discover it from Python, send invoice text, and check the extracted fields with a complete client example.

September 11, 202610 min read
Tutorial

How to Deploy a Python AI Agent Without Kubernetes

A Python AI agent deployed without Kubernetes using YAML, plain Python, deployment-gated tests, Git, and a managed EU runtime. Includes working code.

August 12, 202612 min read
Tutorial

How Small Engineering Teams Can Add an AI Agent to SaaS

A practical, step-by-step path to shipping a first production AI agent with a small team: scope one job, define it in config, connect it to existing systems, and let a runtime handle the rest.

June 12, 20269 min read
Tutorial

Automated Agent Scoring: AI Agent Evaluation with LLM Judges

Automated agent scoring uses an LLM judge to grade sampled or every matching agent run against defined criteria. Track score trends and alert on regressions.

March 29, 202610 min read
Tutorial

Migrate from LangChain to Production AI Agents

A working LangChain prototype still needs to handle real traffic. Migrate existing agent code to a production-grade platform without rewriting from scratch.

March 23, 202611 min read
Tutorial

Database vs Retrieval vs Sessions: Agent Memory Compared

A comparison of Connic's Database, Retrieval, and persistent sessions, including identity, TTL, sanitized events, and runs matched to that identity.

March 4, 202612 min read
Tutorial

AI Agents: From Prototype to Production

A demo works great until 1,000 concurrent users arrive. A practical guide to the production requirements most teams find out about too late.

January 10, 202610 min read
Tutorial

AI Agents in SaaS Without an ML Team

Customers expect AI features even when a product team has no ML engineers. See how teams ship AI agents using skills they already have.

December 5, 20258 min read
Tutorial

AI Agent RAG Tutorial: Retrieval With Citations

Build a production RAG agent with scoped retrieval namespaces, read-only permissions, source citations, custom tool wrappers, and regression tests.

November 15, 20259 min read