Open-monitor adds LLM filtering and signed webhooks to self-hosted tracking
A new self-hosted tool uses large language models to filter search results and page changes, delivering only relevant updates via secure webhooks.
Developer Dhananjay-JSR has released open-monitor, a self-hosted server that watches web searches and specific pages for changes. Published on GitHub in November 2026, the tool uses a large language model to judge whether new content matches a user-defined goal before sending alerts via signed webhooks.
What happened
The project introduces a monitoring system that runs on Node.js or Cloudflare Workers, storing data in SQLite or D1. It supports two primary monitoring types: search monitors that query providers like Tavily or a self-hosted SearXNG instance, and page monitors that fetch URLs and compare text against previous versions. The core innovation is the "judge," an LLM call that scores each candidate result from zero to one based on relevance to a stated goal.
Results scoring at or above 0.5 are kept and forwarded to a configured webhook endpoint. The system handles rate limits for free LLM tiers by sharing per-minute counts across processes and waiting for retry headers when limits are hit. For pages that rely on JavaScript rendering, the tool integrates with headless browsers via the Chrome DevTools Protocol, supporting options like Lightpanda or standard Chrome instances.
Delivery reliability is managed through a robust retry schedule. If a webhook fails, the system attempts redelivery after five seconds, five minutes, thirty minutes, two hours, five hours, and ten hours. Redirects are treated as failures. If all seven attempts fail, the monitor pauses automatically to prevent resource waste, logging the reason as webhook_failing. Secret rotation is supported, with old secrets remaining valid for signing alongside new ones for twenty-four hours.
Key details
- The judge uses OpenRouter free models by default, allowing twenty requests per minute and fifty per day without credits.
- Webhook events follow the Standard Webhooks specification, using HMAC-SHA256 signatures for verification.
- Page diffs sent to the LLM are capped at twenty-four kibibytes to manage token usage.
- The dashboard requires three environment variables to activate: user, password, and session secret.
- Failed login attempts lock the username and IP address for fifteen minutes after five incorrect passwords.
- Telemetry exports traces and counters via OTLP/HTTP, ensuring secrets and page content are never included in logs.
Background
Self-hosted monitoring tools typically rely on keyword matching or simple diffing algorithms to detect changes. These methods often generate significant noise, alerting users to minor layout shifts or irrelevant mentions. By integrating a large language model as a filter, this tool aims to reduce false positives by understanding the semantic context of the change rather than just detecting textual differences.
Signed webhooks are a security standard that ensures the data received by a server actually comes from the expected source. Without signature verification, attackers could send fake alerts to trigger automated actions. The Standard Webhooks protocol used here provides a consistent way to verify these signatures across different programming languages and platforms, enhancing the security of automated workflows.
Why it matters
For teams running their own software infrastructure, noise reduction is critical. Alert fatigue causes engineers to ignore important notifications. By filtering search results and page changes through an LLM before they reach human operators or downstream systems, this tool ensures that only high-signal events trigger actions. This is particularly useful for competitive intelligence, tracking regulatory changes, or monitoring specific technical documentation updates.
The self-hosted nature of the project addresses data privacy concerns common in enterprise environments. Since the monitoring logic, database, and LLM interactions can run entirely within a private network or on trusted cloud infrastructure, sensitive search queries and page contents do not need to pass through third-party SaaS monitoring platforms. This control is vital for companies handling proprietary information or operating under strict compliance requirements.
Additionally, the built-in retry logic and pause mechanisms protect internal services from being overwhelmed by failing endpoints. In complex microservice architectures, a misconfigured webhook receiver can cause cascading issues if the sender does not handle failures gracefully. The explicit handling of redirects as failures and the progressive backoff strategy provide a stable integration pattern for DevOps teams.
What you can do
- Deploy the Docker Compose stack to test the full suite, including SearXNG for private search and Lightpanda for JavaScript rendering.
- Configure the LLM judge to use a local Ollama instance if you require complete air-gapped operation without external API calls.
- Set up OpenTelemetry exporters to send traces to Grafana Cloud, allowing you to visualize the latency of search, judging, and delivery steps.
- Use the "Copy for AGENTS.md" feature in the dashboard to quickly integrate the API documentation into your AI agent’s context window.
- Test webhook verification using the provided Node.js snippet to ensure your receiving endpoint correctly validates HMAC signatures before processing events.
- Monitor the
/v1/monitors/{id}/deliveriesendpoint to audit failed deliveries and identify persistent connectivity issues with your webhook targets.



