llmtee offers self-hosted LLM cost tracking and budget caps

Released on November 11, 2026, llmtee is a new self-hosted proxy designed to monitor and control costs for Large Language Model (LLM) usage. Created by developer garritfra, this tool sits between your applications and providers like OpenAI, Anthropic, or local models such as Ollama, providing visibility into token consumption and enforcing hard budget limits without requiring code changes in your client applications.

What happened

The project introduces a lightweight proxy that functions similarly to the Unix tee command. It passes API requests and responses through to their destination unchanged while simultaneously keeping a copy of the data for analysis. This allows developers to continue using standard SDKs and tools, such as Claude Code or the OpenAI Python library, by simply changing the base URL configuration to point to their local llmtee instance. The tool logs every request, capturing details including token counts, monetary cost, latency, and time to first token.

llmtee is distributed as a single Go binary, which simplifies deployment for small teams and individuals. It uses SQLite by default, meaning a backup is as simple as copying a file, though it also supports PostgreSQL for more robust setups. The current release is version 0.x, indicating that the configuration and features may still undergo changes as the project matures. It is explicitly not designed as a full-featured gateway with routing, fallbacks, or multi-tenancy, focusing instead on cost visibility and control.

Key details

• Budget enforcement: Each API key can be assigned daily and monthly spending limits. When a limit is reached, llmtee returns an HTTP 429 error and stops forwarding requests until the period resets.
• Key management: The system supports two key types: provider keys (forwarded directly) and llmtee keys (which swap in the real provider key server-side). Provider keys are stored only as SHA-256 hashes.
• Low overhead: Benchmarks on an Apple M4 Pro showed added latency of 0.14 ms at p50 and 0.02 ms at p99 under load, with memory usage around 33 MB.
• Data export: Usage data can be accessed via a web UI, CSV exports, Prometheus metrics, or an MCP (Model Context Protocol) server for integration with AI agents.
• Privacy controls: Body logging is enabled by default with a 30-day retention period, but users can disable it globally or per-request to protect sensitive data.
• Provider support: It works with OpenAI, Anthropic, OpenRouter, Groq, and OpenAI-compatible endpoints for local models like Ollama and vLLM.

Background

For teams integrating generative AI into their workflows, cost management is often opaque. Cloud providers charge based on token usage, which can fluctuate wildly depending on prompt length and model choice. Without a proxy, developers must rely on post-hoc billing reports from the provider, making it difficult to set immediate limits or attribute costs to specific users or projects. A proxy server intercepts these network calls, allowing for real-time inspection and modification of traffic before it reaches the external API.

llmtee distinguishes itself from heavier platforms like Langfuse or Helicone by minimizing infrastructure requirements. While other solutions may require multiple containers, databases like ClickHouse, and complex worker queues, llmtee aims to run as a single process. This makes it attractive for small teams that want to avoid the operational overhead of maintaining a large observability stack just to track API spend.

Why it matters

For DevOps engineers and IT leads managing self-hosted software, controlling variable costs is critical. Unchecked LLM usage can lead to unexpected bills, especially when testing new agents or automation scripts. By placing llmtee in the request path, teams can enforce hard caps that prevent budget overruns. Although the tool notes that a single request might overshoot the limit because cost is calculated after the response arrives, it provides a much tighter control loop than manual monitoring.

The ability to attribute costs to specific keys or users helps in internal chargebacks or project accounting. Since llmtee allows tagging requests with custom properties, engineering managers can see exactly which team or application is driving usage. This granularity is essential for optimizing resource allocation and identifying inefficient prompts or redundant calls within a development environment.

Furthermore, the tool’s support for local models alongside hosted APIs provides a unified view of AI spend. Teams experimenting with open-weight models on their own hardware can compare those costs against cloud providers in a single dashboard. This holistic view supports better strategic decisions about where to run inference workloads based on both performance and financial impact.

What you can do

• Deploy llmtee using Docker or a standalone binary to test its impact on your existing LLM workflows without altering application code.
• Configure daily and monthly budgets for each API key to prevent accidental overspending during development or testing phases.
• Enable body logging temporarily to debug high-cost requests, then disable it or use redaction fields to maintain data privacy compliance.
• Integrate the Prometheus metrics endpoint with your existing monitoring stack to alert on unusual spikes in token usage or latency.
• Use llmtee keys for automated agents and CI/CD pipelines to keep provider credentials secure and isolate their spending from human users.
• Export usage data to CSV regularly for financial reporting and to analyze long-term trends in model efficiency and cost per task.