Building a private AI search stack with SearXNG and local LLMs

A developer has released PrivaSearch, an open-source tool that synthesizes web search results into AI-generated answers while keeping all data local. Published on November 8, 2026, the project uses SearXNG for metasearch and Ollama for large language model inference, ensuring user queries never leave the self-hosted environment.

What happened

PrivaSearch is built as a four-container Docker stack running on a single internal network. The architecture isolates the frontend, which serves a React interface via nginx, from the backend components. The API layer, written in Node.js and Express, orchestrates communication between the user interface, the SearXNG metasearch engine, and the local LLM served by Ollama. Only the frontend exposes a host port, typically 8080, while the other services remain unreachable from outside the Docker network.

The system separates the search process from the answer generation. When a user submits a query, the browser first requests paginated results from SearXNG. It then sends the top results to the API, which constructs a prompt for the local LLM. The model streams a synthesized answer back to the client, citing sources with bracketed numbers. This separation allows users to regenerate answers with different source counts without re-querying the web engines.

To maintain strict privacy, the project minimizes dependencies and controls data persistence. The API relies only on Express and Node’s built-in fetch capability. Queries and results exist only in process memory during the request and are not stored on disk. The only persistent data includes the model weights and SearXNG configuration. Optional features like page summarization and geo-IP lookups are disabled by default to prevent extra outbound requests.

Key details

• The stack runs four containers: frontend (nginx/React), API (Node/Express), SearXNG, and Ollama on an internal Docker network.
• Search and answer generation are separate steps, allowing users to adjust source counts without re-running the initial search.
• The system prompt restricts the LLM to use only provided results, cite claims with [n], and avoid outside knowledge.
• Default temperature is set to 0.2 for factual consistency, and the model keeps resident for 30 minutes to reduce load times.
• Optional Tor or VPN tunnels can hide the server’s IP from upstream search engines, with Gluetun providing a kill switch for the VPN option.
• The default model is qwen2.5:1.5b, which requires about 1 GB of RAM and delivers warm answers in approximately 8 seconds on CPU.

Background

Retrieval-Augmented Generation (RAG) is a technique where an AI model answers questions using specific external documents rather than just its training data. In this setup, SearXNG acts as the retrieval engine, fetching live web results. Ollama then processes these results to generate a coherent summary. By running both components locally, users avoid sending sensitive search terms to third-party AI providers.

SearXNG is a privacy-respecting metasearch engine that aggregates results from multiple providers like Google and Bing without tracking users. Ollama is a tool that simplifies running large language models on local hardware. Combining them allows for a search experience that feels modern and intelligent but remains under the user’s full control, addressing growing concerns about data leakage in cloud-based AI services.

Why it matters

For teams that self-host their infrastructure, maintaining privacy while adopting AI tools is a significant challenge. Public AI APIs often log queries for training or improvement, which may violate internal compliance policies or expose proprietary information. PrivaSearch demonstrates that it is possible to get cited, synthesized answers without sacrificing data sovereignty. This approach is particularly relevant for organizations handling sensitive client data or intellectual property.

The architecture also highlights the importance of minimal dependencies in security-conscious projects. By relying on Node’s built-in capabilities and avoiding heavy frameworks, the attack surface is reduced. This makes the code easier to audit and maintain, a critical factor for sysadmins who need to trust the software they deploy. The explicit handling of edge cases, such as model failures or empty results, ensures a reliable user experience even when components are under stress.

Furthermore, the optional tunneling features address the metadata privacy gap. Even if the query content is safe, the IP address making the request can reveal organizational identity. Routing traffic through Tor or a VPN with a kill switch ensures that upstream engines see only anonymous traffic. This level of detail shows a mature understanding of privacy threats beyond just content encryption.

What you can do

• Deploy the stack using Docker Compose by running npm run setup and docker compose up -d to test the local AI search workflow.
• Adjust the OLLAMA_KEEP_ALIVE setting to balance between response speed and memory usage based on your server’s capacity.
• Enable the Tor or VPN options if hiding your server’s IP from search engines is a requirement for your threat model.
• Review the prompt.js file to understand how the system constrains the LLM, and modify it if you need different citation styles or behaviors.
• Disable optional features like ALLOW_PAGE_FETCH and SHOW_LOCATION in the configuration to ensure no unexpected outbound requests occur.
• Monitor resource usage when switching to larger models like llama3.1:8b, as they provide better answers but require significantly more CPU power and time.