Self-hosting

Hidden costs in self-hosted search and AI pipelines on Azure

A solo developer shares a detailed breakdown of running a clinical-trials search site, revealing unexpected bandwidth spikes and AI token waste.

Illustration of a server rack with warning lights and leaking cables over a bill
Illustration created for this article

Oleksandr Gamaniuk, the sole developer behind the clinical-trials search platform hipa.ai, published a detailed cost analysis of his infrastructure on November 17, 2026. The report covers seven months of operation on Microsoft Azure, highlighting how shared resources, memory constraints, and unmonitored AI usage created financial and operational surprises for a small, self-hosted application.

What happened

Gamaniuk began by auditing his Azure Cost Management data to answer frequent questions about the expense of running a site with over 600,000 trial records and weekly AI-generated content. He found that while direct costs for storage and logging were negligible, shared compute plans and network egress fees dominated the bill. The application initially ran on a shared Windows App Service plan and a minimal Linux virtual machine for Elasticsearch, leading to ambiguous cost allocation and recurring stability issues.

The most significant operational failure involved the Elasticsearch instance, which ran on a VM with only 3.5 GB of RAM. The Java process frequently consumed nearly all available memory, causing the Linux kernel’s out-of-memory killer to terminate the service multiple times between April and July 2026. Because the system service was not configured to restart automatically, the search engine remained offline until manual intervention occurred, breaking cache-miss renders for users despite other parts of the site remaining accessible via Cloudflare.

In an effort to reduce compute costs, Gamaniuk moved the search engine to a self-hosted Linux machine outside of Azure, connecting it via a WireGuard tunnel. While this eliminated the virtual machine compute charge, it introduced a new and substantial cost: network egress. In just eleven days, the bandwidth required to tunnel search queries and responses back into Azure cost more than the previous month’s entire compute bill for that service, demonstrating how architectural changes can shift expenses to unexpected line items.

Key details

  • The Elasticsearch VM (1 vCPU, 3.5 GB RAM) crashed five times due to memory exhaustion, with the Java process using approximately 2.6 GB of RSS.
  • Bandwidth costs for web egress exceeded the combined costs of blob storage, Key Vault, and Log Analytics, driven largely by bot traffic.
  • Moving search off Azure reduced compute costs to zero but increased monthly egress spending to $42.30 in October due to WireGuard tunnel traffic.
  • Deployments on Windows App Service took up to 75 minutes due to slow SMB file transfers, but dropped to six seconds after switching to run-from-package mode.
  • AI generation costs were untracked for seven months until logging was added in late October, revealing that provider choice significantly impacted output length and token usage.
  • A Google search algorithm update in July 2026 reduced daily traffic by 92%, representing a hidden cost that outweighed the unmeasured AI spending.

Background

Self-hosting infrastructure often involves trade-offs between control and complexity. In this case, the developer used a "standalone" build of Next.js, which separates the application from the framework code to reduce deployment size. However, on Windows-based cloud services, deploying thousands of small files over network shares can be notoriously slow compared to Linux-based container deployments. Additionally, Elasticsearch is a Java-based search engine known for high memory consumption. Running it on a minimal virtual machine without swap space or automatic restart policies creates a fragile system where any minor spike in operating system activity can crash the primary service.

Network egress refers to data leaving a cloud provider’s network. Many developers focus on compute and storage costs but overlook egress fees, which can accumulate quickly when applications communicate across different networks or regions. In this scenario, tunneling search results from an external server back into Azure triggered these fees, illustrating that "cheaper" compute elsewhere may result in higher total costs if data transfer volumes are high.

Why it matters

For teams running their own software, this case study highlights the danger of shared infrastructure. While sharing a premium app service plan reduces the headline cost, it obscures the true unit economics of individual projects. Without precise cost allocation, it becomes difficult to justify optimizations or predict scaling expenses. The experience also serves as a reminder that stateful services like databases and search engines require careful resource provisioning. Assuming a service will fit into the smallest available tier often leads to instability that requires more engineering time to fix than the saved compute costs justify.

The AI component of the story underscores the importance of observability in generative workflows. Without tracking token usage from day one, developers cannot accurately measure the return on investment for automated content generation. Furthermore, the traffic drop following the publication of thousands of programmatic pages suggests that search engines may penalize sites that appear to prioritize scale over quality. This "hidden cost" of lost organic traffic can far exceed the direct infrastructure bills, affecting the long-term viability of content-heavy self-hosted applications.

What you can do

  • Configure systemd services with Restart=on-failure to ensure critical components like Elasticsearch recover automatically after crashes.
  • Monitor network egress costs closely when moving services outside your primary cloud provider, especially when using tunnels or proxies.
  • Implement token logging for all AI API calls from the start of development to track usage patterns and detect billing anomalies early.
  • Use run-from-package deployment methods to avoid slow file-by-file transfers and reduce deployment times from minutes to seconds.
  • Validate external dependencies, such as image generation, before triggering expensive AI tasks to prevent wasted tokens on failed processes.
  • Regularly audit shared infrastructure bills to isolate project-specific costs and identify unexpected spikes in usage or pricing.

More news

All news