Sovereign RAG Deployment Patterns: From Air-Gapped to Edge

Sovereign RAG Deployment Patterns

Architecting Retrieval-Augmented Generation for Every Trust Level -- From Completely Disconnected to Fully Portable

📄 Technical Reference ⏱️ 35 min read 💻 Deployment Architecture

A sovereign RAG system is not one architecture. It is a family of architectures, each trading operational convenience for data control. The fully air-gapped deployment -- a server in a locked room with no network connection, ingesting documents via USB -- is the most sovereign and the most burdensome. The edge deployment -- a RAG stack on a mini PC in a backpack -- is the least sovereign and the most flexible. Between these poles sit three patterns that cover the majority of real-world deployments: isolated LAN for organizations that need internal access without internet exposure, hybrid for organizations with mixed data sensitivity, and local-first for users who want sovereignty without sacrificing the convenience of occasional internet access. This guide maps the full spectrum, provides reference architectures and Docker Compose configurations for each pattern, explains how to move documents and model updates into air-gapped systems, details backup and monitoring strategies that work without internet, and gives a concrete decision framework for choosing the right pattern for your constraints. Every pattern described here has been deployed in production by organizations ranging from government agencies to solo researchers.

Deployment PatternsAir-GappedIsolated LANHybrid RAGLocal-FirstEdge RAGData SovereigntyDocker Compose

🎯 Bottom Line: There is no single "right" sovereign RAG deployment. The correct pattern depends on your data sensitivity, your user count, your remote access needs, your budget, and your maintenance capacity. For most organizations, the answer is either isolated LAN (sensitive data, multiple users, no internet needed) or local-first (sensitive data, one to five users, internet tolerated for updates). Air-gapped is reserved for genuinely classified environments. Hybrid adds complexity that is only justified when you have a clear split between sensitive and non-sensitive document sets. Edge is for single users who need portability. Start with the simplest pattern that satisfies your constraints, and move toward more sovereign patterns only if your threat model demands it.

The Spectrum of Sovereignty

Sovereignty is not binary. It is a spectrum, and every RAG deployment sits somewhere on it. At one end is the fully cloud-hosted SaaS RAG service -- your documents are uploaded to a third-party API, embeddings and vectors live in someone else's database, and the language model runs on someone else's GPU. You have no control over data retention, no insight into who can access your queries, and no recourse if the provider changes their terms. At the other end is the air-gapped system -- a server in a physically secured room, no network connection of any kind, powered by a dedicated electrical circuit, with documents arriving on USB drives that are scanned and verified before ingestion. Between these extremes, sovereignty is a function of five variables:

Variable	More Sovereign	Less Sovereign
Network connectivity	No network (air-gapped)	Persistent internet
Hardware ownership	Owned, in your facility	Leased or rented cloud instances
Data location	On your disk, never leaves	Stored in cloud regions you do not control
Model provenance	Signed, verified, offline model files	Pulled from a registry at runtime
Access control	Physical + network + application layers	Application-layer only

The five deployment patterns in this guide correspond to five points on this spectrum. They are not arbitrary -- each one represents a meaningfully different threat model and a different set of trade-offs. The diagram below shows where each pattern sits relative to the others, along with the typical use cases that fall into each band.

SOVEREIGNTY SPECTRUM (most sovereign → least sovereign): [AIR-GAPPED] [ISOLATED LAN] [HYBRID] [LOCAL-FIRST] [EDGE] [CLOUD SaaS] | | | | | | No network LAN only, no Local RAG for RAG local, RAG on one Documents at all internet sensitive, internet device, uploaded to VPN for remote cloud for optional for portable third-party access public docs updates API | | | | | | Classified Medical Mixed-data Home users, Personal Least government, practices, businesses small use, sovereign, extreme legal firms, businesses, field no data medical financial researchers work control privacy, institutions patent research | | | | | MOST SOVEREIGN LEAST SOVEREIGN

💡 Where This Guide Focuses: This guide covers the five on-premise patterns -- air-gapped through edge. The fully cloud SaaS pattern is excluded because it is, by definition, not sovereign. If your data can be sent to a third-party API, you do not need a deployment pattern guide; you need an API integration guide. The entire premise of sovereign RAG is that your data never leaves hardware you control. The patterns below are the architectures that make that possible at every scale and every threat level.

Pattern 1: Air-Gapped (Maximum Sovereignty)

1The air-gapped pattern is the gold standard of data sovereignty. The RAG server has no network connection -- no Ethernet, no WiFi, no Bluetooth, no cellular modem. It is a machine that exists in isolation, communicating with the outside world only through physical media: USB drives, external hard drives, or optical discs that are manually carried to and from the machine. This pattern is used by intelligence agencies for classified document analysis, by healthcare organizations handling data so sensitive that even encrypted network transmission is unacceptable, and by research institutions protecting patent-pending intellectual property where any data leak could cost millions.

Architecture

The air-gapped RAG stack is the full sovereign stack -- Qdrant, Postgres, Ollama, embedding service, re-ranker, Tika, Tesseract, Whisper, orchestrator, and web UI -- running entirely on one or more servers in a physically secured room. All services run in Docker containers, but Docker images must be pre-loaded onto the machine from offline media; no image can be pulled at runtime. The GPU (if used for embeddings or LLM inference) is installed directly in the server. A local NTP server (a small hardware device) keeps the clock accurate for audit logs, since internet NTP is unavailable. A hardware security module (HSM) or Trusted Platform Module (TPM) stores encryption keys for the data at rest.

Hardware Requirements

Component	Specification	Purpose
Server	32+ cores, 128 GB RAM, 4 TB NVMe SSD, 1× RTX 4090 or A6000 (24 GB VRAM)	Runs all RAG services
HSM / TPM	YubiHSM 2 or TPM 2.0 on motherboard	Stores encryption keys for data at rest
Local NTP device	GPS-synchronized NTP appliance	Accurate clock for audit logs without internet
USB / external drive	USB 3.2 external SSD, 1-4 TB	Sole data ingress and egress channel
UPS	1500 VA minimum, 30+ min runtime	Clean shutdown on power loss

Network Configuration

There is no network configuration. The server's Ethernet ports are disabled at the BIOS level. WiFi is disabled (or the card is physically removed). Bluetooth is disabled. USB ports are the only external interface, and those can be restricted via USBGuard policy to only allow whitelisted devices by serial number. The server has no IP address, no hostname resolution, no DNS. All inter-service communication happens over Docker's internal bridge network on localhost.

⚠️ Critical Trade-Offs No remote access. Updates are manual -- a new model or software version must be downloaded on a separate internet-connected machine, verified, written to USB, scanned for malware, and physically carried to the air-gapped server. Maintenance requires physical presence. Backups must go to external drives, not network storage. Monitoring is local only -- no email or Slack alerts, no cloud dashboards. This is the hardest pattern to maintain, and it should only be chosen when the data sensitivity genuinely justifies the operational burden.

Best Suited For

Classified government document analysis (intelligence, defense, diplomatic)
Extreme medical privacy cases (genetic data, mental health records, HIV status)
Patent-pending research where any leak could cost millions
Legal matter involving national security or trade secrets at that level
Any environment where network exfiltration is an existential threat

Pattern 2: Isolated LAN (High Sovereignty)

2The isolated LAN pattern connects the RAG server to a local network but not to the internet. Users on the same LAN can access the RAG web UI directly. Remote users connect via a VPN that terminates inside the LAN. The RAG server itself has no route to the internet -- firewall rules block all outbound traffic from the RAG VLAN. This pattern is the workhorse of sovereign deployments for organizations: it provides the convenience of network access for multiple users while maintaining strong data sovereignty. Medical practices, legal firms, and financial institutions are the primary users of this pattern.

Architecture

The full RAG stack runs on a server placed on an isolated VLAN. A firewall sits between the RAG VLAN and the rest of the network, allowing only the specific ports needed for the web UI and API (typically 8080 for the web UI and 8000 for the API). A VPN gateway (OpenVPN Access Server or WireGuard) provides remote access for users outside the office. The VPN terminates on a separate VLAN and routes into the RAG VLAN. All outbound traffic from the RAG VLAN is blocked -- the server cannot reach the internet even if a service is misconfigured to attempt it. Updates are performed by a maintenance administrator who temporarily connects a USB drive with pre-downloaded, verified packages, or by a controlled maintenance window where outbound access to specific, whitelisted update servers is temporarily enabled.

Hardware Requirements

Component	Specification	Purpose
RAG Server	16-32 cores, 64-128 GB RAM, 2-4 TB NVMe SSD, 1× RTX 4090 (24 GB VRAM)	Runs all RAG services
VPN Gateway	Small appliance or VM (4 cores, 4 GB RAM)	WireGuard or OpenVPN for remote access
Firewall	pfsense or OPNsense appliance, or iptables rules on host	Blocks outbound from RAG VLAN
NAS (optional)	4-8 TB, RAID 5 or 6, on RAG VLAN	Network backup target
Managed switch	VLAN-capable, L3 switch	Separates RAG VLAN from office LAN

Network Configuration

The RAG server lives on its own VLAN (for example, 192.168.10.0/24). The firewall allows inbound TCP 8080 (web UI) and TCP 8000 (API) from the office LAN and VPN VLAN. All outbound traffic from the RAG VLAN is denied by default. If updates are needed, a maintenance window is declared, an explicit firewall rule allowing traffic to a specific update server IP is added temporarily, the update is performed, and the rule is removed. This is documented in the change log. Inter-service communication within the RAG server uses Docker's internal bridge network.

💡 VPN Access Design: The VPN should use certificate-based authentication, not passwords. Each user gets a unique client certificate. The VPN gateway logs all connections (user, time, source IP) to syslog. VPN access is restricted to the RAG VLAN only -- VPN users cannot route to the internet through the office network. This prevents the VPN from becoming an unintended data exfiltration channel.

Best Suited For

Medical practices (HIPAA-compliant document Q&A across multiple practitioners)
Legal firms (attorney work product, privileged communications)
Financial institutions (trading strategies, client portfolios, internal research)
Corporate R&D departments (proprietary research, internal technical documentation)
Any organization with multiple users who need concurrent access to sensitive documents

Pattern 3: Hybrid (Moderate Sovereignty)

3The hybrid pattern runs two RAG systems in parallel: a local, sovereign RAG stack for sensitive documents, and a cloud-hosted RAG service for non-sensitive public documents. A query router in front of both systems inspects each query and decides which system(s) to send it to. Sensitive queries go to the local stack only. Public-knowledge queries go to the cloud stack. Queries that might span both go to both, and the results are merged. This pattern is for organizations that have a clear split between sensitive and non-sensitive document sets and want the cost savings and scalability of cloud for the non-sensitive portion.

⚠️ The Critical Risk in Hybrid Deployments The fundamental risk in a hybrid RAG system is that sensitive data crosses into the cloud half. This can happen in two ways: the query router misroutes a sensitive query to the cloud, or a user phrases a query that embeds sensitive information (for example, "What does the contract with Acme Corporation say about liability?" where the contract is sensitive but the company name itself is not). The query router must use content inspection -- scanning the query for sensitive patterns, entity names from the sensitive corpus, or keywords flagged by the data classification policy -- to make routing decisions. Even with content inspection, the hybrid pattern adds a class of risk that does not exist in pure-local deployments. Only choose hybrid if you have a genuine, large, non-sensitive corpus where cloud economics matter.

Architecture

The local half is identical to the isolated LAN or local-first pattern (depending on whether multiple users need access). The cloud half is a separate RAG deployment on a cloud provider (AWS, GCP, Azure) or a cloud RAG SaaS service. A query router service sits in front of both. The router inspects each incoming query, applies classification rules, and routes accordingly. Results from both halves are merged by a fusion service that deduplicates and re-ranks the combined results. The fusion service runs locally -- cloud results are pulled back to the local side before being shown to the user, ensuring that the final response (including the LLM generation) happens on local hardware.

Hardware Requirements

The local half requires the same hardware as the isolated LAN or local-first pattern. The cloud half is provisioned on the cloud provider and scales with demand. The query router and fusion service are lightweight (Python services, 2 cores and 4 GB RAM each) and run on the local server alongside the RAG stack. The primary hardware consideration is ensuring the local server has enough GPU capacity for the sensitive query load plus the fusion and final LLM generation for all queries.

Network Configuration

The local server has a controlled outbound connection to the cloud RAG API endpoint only. Firewall rules restrict outbound traffic to the specific IP ranges and ports used by the cloud RAG service. No other outbound traffic is allowed. The connection to the cloud RAG service uses TLS with certificate pinning to prevent man-in-the-middle interception. The query router and fusion service are the only components that initiate outbound connections; the RAG stack itself (Qdrant, Postgres, Ollama) has no outbound access.

Best Suited For

Businesses with mixed document types (sensitive internal + public regulatory)
Engineering firms (proprietary design docs + public standards and specifications)
Law firms (privileged case files + public case law and statutes)
Healthcare organizations (patient records + published medical literature)
Any organization where the non-sensitive corpus is large enough that cloud economics matter

Pattern 4: Local-First with Optional Internet (Pragmatic Sovereign)

4The local-first pattern is the pragmatic sovereign choice. The RAG stack runs entirely on a local server or mini PC. Internet access is available but not required for operation -- it is used for model downloads, software updates, and optional remote monitoring. All queries, all document ingestion, all embedding, all retrieval, and all LLM generation happen locally. The internet connection is a maintenance channel, not a query path. This is the most common pattern for home users, small businesses, and independent researchers who want data sovereignty without the operational overhead of network isolation.

Architecture

The full RAG stack runs on a single machine -- typically a mini PC (Intel NUC, Beelink, Mac Mini) or a standard tower server. All services run in Docker containers. The machine has an internet connection, but firewall rules restrict outbound traffic to specific destinations: the Docker registry for image updates, the Ollama model library for model downloads, the Debian/Ubuntu package repositories for OS updates, and optionally a monitoring service. The web UI is accessed via the machine's local IP address on the LAN. There is no VPN -- if remote access is needed, it is added on top of this pattern using a simple WireGuard tunnel.

Hardware Requirements

Component	Specification	Purpose
Mini PC (small corpus)	8 cores, 32 GB RAM, 1 TB NVMe, integrated GPU or 8 GB discrete GPU	Up to 10K documents, CPU or small GPU inference
Standard server (medium corpus)	16 cores, 64 GB RAM, 2 TB NVMe, 1× RTX 4090 (24 GB VRAM)	Up to 100K documents, GPU inference
UPS (recommended)	1000 VA, 15+ min runtime	Clean shutdown on power loss

Network Configuration

The server has a static IP on the LAN. Inbound access is on port 8080 (web UI) and 8000 (API) from the LAN. Outbound access is restricted by firewall rules to a whitelist of update destinations. The simplest implementation is iptables rules on the host:

# Outbound firewall rules: allow updates only, block everything else
# Allow DNS (needed for name resolution)
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -p tcp --dport 53 -j ACCEPT

# Allow Docker registry (for image updates)
iptables -A OUTPUT -d registry-1.docker.io -p tcp --dport 443 -j ACCEPT

# Allow Ollama model library (for model downloads)
iptables -A OUTPUT -d ollama.com -p tcp --dport 443 -j ACCEPT
iptables -A OUTPUT -d registry.ollama.ai -p tcp --dport 443 -j ACCEPT

# Allow OS package repos (for security updates)
iptables -A OUTPUT -d deb.debian.org -p tcp --dport 443 -j ACCEPT
iptables -A OUTPUT -d security.ubuntu.com -p tcp --dport 443 -j ACCEPT

# Block everything else outbound
iptables -A OUTPUT -j DROP

⚠️ The Internet Is an Attack Surface The local-first pattern's internet connection, even restricted to updates, is an attack surface. A compromised update server (supply chain attack) could deliver a malicious package or model. Mitigate this by: (1) verifying GPG signatures on all packages and model files, (2) pinning to specific model hashes so that a changed model is rejected, (3) running the RAG stack as non-root in Docker with read-only mounts where possible, and (4) auditing outbound connections regularly. If your threat model includes state-level actors, move to isolated LAN or air-gapped.

Best Suited For

Home users wanting a personal RAG assistant over their own documents
Small businesses (1-10 employees) with sensitive internal documents
Independent researchers with proprietary datasets
Consultants who need RAG over client data but cannot use cloud services
Anyone who wants sovereignty with minimal operational overhead

Pattern 5: Edge / Embedded (Minimal Footprint)

5The edge pattern puts the entire RAG stack on a single portable device -- a laptop, a mini PC, or even a Raspberry Pi 5. All services run on one device. There is no separate server, no network infrastructure, no VLANs. The device may connect to WiFi for convenience, but the RAG stack operates entirely on localhost. This pattern trades scale and multi-user access for portability. It is the pattern for individuals who need RAG in the field, on the road, or in environments where deploying a server is not practical.

Architecture

The RAG stack runs as Docker containers on a single device. For a mini PC (Intel NUC, Beelink), the full stack can run with a smaller embedding model and a 7B-8B parameter LLM. For a Raspberry Pi 5 (8 GB RAM), the stack uses quantized models (GGUF Q4_K_M) and a lightweight embedding model (all-MiniLM-L6-v2), and may omit Whisper and Tesseract to save resources. The web UI is accessed via localhost:8080 in a browser on the same device. For field work, a phone or tablet on the same WiFi hotspot can access the UI.

Hardware Requirements

Device	Specification	Corpus Capacity	LLM Size
Laptop with GPU	8-core CPU, 32 GB RAM, 1 TB SSD, RTX 4060 (8 GB VRAM)	Up to 50K documents	8B (Q4_K_M)
Mini PC (Intel NUC / Beelink)	8-core CPU, 32 GB RAM, 1 TB NVMe, integrated GPU	Up to 10K documents	7B (Q4, CPU)
Raspberry Pi 5	4-core CPU, 8 GB RAM, 256 GB NVMe HAT, no GPU	Up to 2K documents	3B (Q4, CPU)

Network Configuration

Minimal. The device runs Docker with the default bridge network. All services communicate over localhost. The web UI is served on localhost:8080. If the device connects to WiFi (for model downloads or to allow a phone to access the UI), a basic firewall blocks all inbound connections except from the local network. For field work where no WiFi is available, a phone hotspot provides the local network, and the RAG device connects to it.

💡 Optimizing for Small Devices: On a Raspberry Pi 5, use Qdrant with scalar quantization (8-bit) to reduce memory usage. Use a 384-dimensional embedding model (all-MiniLM-L6-v2) instead of 768-dimensional models. Use a 3B parameter LLM (Llama 3.2 3B or Phi-3 mini) in GGUF Q4_K_M format. Omit Whisper and Tesseract unless specifically needed. Set Qdrant's ef_construct to 128 (down from 256) to reduce index build time. These compromises make RAG possible on 8 GB of RAM, though query latency will be 5-15 seconds per query rather than 1-5 seconds.

Best Suited For

Personal use (RAG over personal notes, research, documents)
Traveling professionals (lawyer on the road, doctor in remote clinic)
Field work (geologist, biologist, archaeologist with offline reference docs)
Demo and prototyping (show RAG capability without server infrastructure)
Education (teaching RAG concepts on a single device)

Hardware Reference Architectures

Each deployment pattern has a reference hardware architecture. These are not the only valid configurations, but they are known-good starting points. Adjust based on your corpus size, query volume, and performance requirements. The GPU is the most expensive and most impactful component -- if in doubt, invest in GPU over CPU. The tables below specify complete builds for each pattern, from the air-gapped server room to the pocket-sized Raspberry Pi.

Air-Gapped Reference Architecture

Component	Specification	Notes
Server	Dell PowerEdge R760 or equivalent, 32-core Xeon, 128 GB DDR5 ECC, 2× 4 TB NVMe SSD (RAID 1), 1× NVIDIA A6000 (48 GB VRAM)	Redundant power supplies
Physical security	Dedicated server room, badge access, camera surveillance, no windows	Room itself is part of the security boundary
Network	No network ports enabled at BIOS. WiFi card physically removed. USB ports restricted via USBGuard	Network is absent, not just disabled
Key storage	YubiHSM 2 hardware security module, or TPM 2.0 on motherboard	Encryption keys never leave the HSM
NTP	GPS-synchronized NTP appliance (e.g., Meinberg LANTIME)	Accurate timestamps for audit logs
UPS	APC Smart-UPS 1500 VA, 30+ min runtime	Enables clean shutdown
Console	Direct-connected monitor and keyboard (no KVM over IP)	Physical access for administration

Isolated LAN Reference Architecture

Component	Specification	Notes
RAG server	16-32 core CPU, 64-128 GB RAM, 2-4 TB NVMe SSD, 1× RTX 4090 (24 GB VRAM)	Handles up to 100K documents, 20-50 concurrent users
VLAN infrastructure	Managed L3 switch (e.g., Mikrotik CRS326, Ubiquiti EdgeSwitch)	Separates RAG VLAN from office VLAN
Firewall	Netgate (pfSense) appliance or VyOS VM	Blocks outbound from RAG VLAN, allows inbound 8080/8000
VPN gateway	WireGuard on a small VM or appliance (4 cores, 4 GB RAM)	Certificate-based auth, logs all connections
NAS (backup)	Synology DS923+ or TrueNAS server, 8 TB RAID 5, on RAG VLAN	Nightly backups of Qdrant, Postgres, files
UPS	APC Smart-UPS 1500 VA	Clean shutdown on power loss

Local-First Reference Architecture

Component	Specification	Notes
Mini PC (small)	Beelink SER8 or Intel NUC 13 Pro, 8-core Ryzen/Intel, 32 GB RAM, 1 TB NVMe, integrated GPU	Up to 10K documents, CPU inference (7B LLM Q4)
Tower server (medium)	Custom build or workstation, 16-core CPU, 64 GB RAM, 2 TB NVMe, RTX 4090 (24 GB VRAM)	Up to 100K documents, GPU inference
Firewall	iptables/nftables on host, outbound restricted to update servers	See code block in Pattern 4 above
Backup	External USB drive (4 TB), nightly rsync	Offline backup, stored separately
UPS (optional)	CyberPower 1000 VA	Recommended but not critical for home use

Edge Reference Architecture

Device	Specification	Estimated Cost	Capability
Raspberry Pi 5 (8 GB)	4-core ARM Cortex-A76, 8 GB LPDDR4X, 256 GB NVMe HAT, active cooler	$150-$200	2K docs, 3B LLM, 10s/query
Mini PC (Beelink SER8)	8-core Ryzen 7, 32 GB DDR5, 1 TB NVMe, integrated GPU	$600-$800	10K docs, 7B LLM, 5s/query
Laptop with GPU	8-core CPU, 32 GB RAM, 1 TB SSD, RTX 4060 (8 GB VRAM)	$1,500-$2,000	50K docs, 8B LLM, 2s/query
Portable battery pack	100 Wh, 100W USB-C PD output	$100-$150	Field operation without wall power

Docker Compose Deployment Per Pattern

The sovereign RAG stack is the same set of services across all patterns. What changes is which services are included, how images are loaded, and what network access they have. The table below summarizes the service profile for each pattern, followed by the Docker Compose configurations for each.

Service	Air-Gapped	Isolated LAN	Hybrid (local)	Local-First	Edge (Pi)
Qdrant (vector DB)	Yes	Yes	Yes	Yes	Yes (quantized)
Postgres (metadata + FTS)	Yes	Yes	Yes	Yes	Yes
Ollama (LLM)	Yes (GPU)	Yes (GPU)	Yes (GPU)	Yes (GPU/CPU)	Yes (CPU, small)
Embedding service	Yes (GPU)	Yes (GPU)	Yes (GPU)	Yes	Yes (small model)
Re-ranker	Yes	Yes	Yes	Yes	Optional (CPU)
Apache Tika	Yes	Yes	Yes	Yes	Optional
Tesseract (OCR)	Yes	Yes	Yes	Yes	Optional
Whisper (audio)	Yes	Yes	If needed	If needed	No (too heavy)
Redis (queue)	Yes	Yes	Yes	Yes	Yes
Web UI	Yes (local)	Yes (LAN)	Yes	Yes	Yes (localhost)
Prometheus	Yes (local)	Yes	Yes	Yes	Optional
Grafana	Yes (local)	Yes	Yes	Yes	Optional
cAdvisor	Yes	Yes	Yes	Yes	Optional
Query Router (hybrid only)	No	No	Yes	No	No
Fusion Service (hybrid only)	No	No	Yes	No	No
WireGuard VPN	No	Yes (separate)	If needed	If needed	No
Image loading	Offline (pre-loaded)	Pre-loaded or local registry	Pre-loaded or pull	Pull from registry	Pull from registry

Air-Gapped Docker Compose

For air-gapped deployments, all Docker images must be pre-loaded. The standard workflow: on a connected machine, pull all images, save them as tar archives with docker save, transfer to USB, and load on the air-gapped server with docker load. The Compose file references local images only -- no image is pulled at runtime.

# Air-gapped docker-compose.yml -- all images pre-loaded locally
version: '3.8'

services:
  qdrant:
    image: qdrant/qdrant:v1.12.4        # pre-loaded via docker load
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - qdrant_data:/qdrant/storage
    environment:
      - QDRANT__SERVICE__GRPC_PORT=6334

  postgres:
    image: postgres:16-alpine
    ports:
      - "5432:5432"
    volumes:
      - pg_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=rag
      - POSTGRES_USER=rag
      - POSTGRES_PASSWORD_FILE=/run/secrets/pg_password
    secrets:
      - pg_password

  ollama:
    image: ollama/ollama:0.5.4
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  embedding:
    image: sovereign-embedding:latest   # built locally, pre-loaded
    ports:
      - "8081:8080"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  reranker:
    image: sovereign-reranker:latest
    ports:
      - "8082:8080"

  tika:
    image: apache/tika:3.0.0.0-full
    ports:
      - "9998:9998"

  tesseract:
    image: sovereign-tesseract:latest
    ports:
      - "8083:8080"

  whisper:
    image: sovereign-whisper:latest
    ports:
      - "8084:8080"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  orchestrator:
    image: sovereign-orchestrator:latest
    ports:
      - "8000:8000"
    depends_on:
      - qdrant
      - postgres
      - ollama
      - embedding
      - reranker
      - tika
      - redis

  webui:
    image: sovereign-webui:latest
    ports:
      - "8080:80"
    depends_on:
      - orchestrator

  prometheus:
    image: prom/prometheus:v2.55.0
    ports:
      - "9090:9090"
    volumes:
      - prometheus_data:/prometheus
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  grafana:
    image: grafana/grafana:11.3.0
    ports:
      - "3000:3000"
    volumes:
      - grafana_data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD_FILE=/run/secrets/grafana_password
    secrets:
      - grafana_password

secrets:
  pg_password:
    file: /secrets/pg_password.txt
  grafana_password:
    file: /secrets/grafana_password.txt

volumes:
  qdrant_data:
  pg_data:
  ollama_data:
  prometheus_data:
  grafana_data:

Isolated LAN Docker Compose

The isolated LAN Compose file is identical to the air-gapped file, with two differences: (1) images can be pulled from a local Docker registry on the LAN rather than pre-loaded via USB, and (2) the web UI binds to the server's LAN IP address instead of localhost so other machines on the network can access it.

# Isolated LAN: web UI binds to LAN IP, images from local registry
  webui:
    image: 192.168.10.100:5000/sovereign-webui:latest
    ports:
      - "192.168.10.50:8080:80"   # Bind to LAN IP, not 0.0.0.0
    depends_on:
      - orchestrator

# All other services same as air-gapped, but images pulled from
# local registry at 192.168.10.100:5000 instead of pre-loaded
# e.g.: image: 192.168.10.100:5000/qdrant/qdrant:v1.12.4

Hybrid Docker Compose (Local Half)

The local half of a hybrid deployment adds two services: a query router that classifies and routes queries, and a fusion service that merges results from local and cloud. The cloud half is deployed separately (on the cloud provider) and is not in this Compose file.

# Hybrid: adds query router and fusion service
  # (all standard RAG services same as local-first)

  query-router:
    image: sovereign-query-router:latest
    ports:
      - "8090:8090"
    environment:
      - LOCAL_RAG_URL=http://orchestrator:8000
      - CLOUD_RAG_URL=https://cloud-rag.company.example.com/api
      - CLASSIFICATION_RULES=/etc/rag/classification.json
    volumes:
      - ./classification.json:/etc/rag/classification.json
    depends_on:
      - orchestrator

  fusion-service:
    image: sovereign-fusion:latest
    ports:
      - "8091:8091"
    environment:
      - LOCAL_RAG_URL=http://orchestrator:8000
      - CLOUD_RAG_URL=https://cloud-rag.company.example.com/api
      - LLM_URL=http://ollama:11434
    depends_on:
      - ollama
      - orchestrator

  webui:
    image: sovereign-webui:latest
    ports:
      - "8080:80"
    environment:
      - RAG_BACKEND=http://query-router:8090   # UI talks to router, not orchestrator directly
    depends_on:
      - query-router
      - fusion-service

Local-First Docker Compose

The local-first Compose file pulls images from Docker Hub during initial setup. After the first docker compose up, images are cached locally. Subsequent restarts do not require internet. For a mini PC without a GPU, omit the deploy.resources blocks and use CPU-based configurations.

# Local-first: standard images, no GPU blocks if using CPU
  embedding:
    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
    # Use :cpu-1.5 for CPU-only, :1.5 for GPU
    ports:
      - "8081:80"
    # No deploy.resources block for CPU-only mini PC

  ollama:
    image: ollama/ollama:0.5.4
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    # For GPU: add deploy.resources block as in air-gapped
    # For CPU: omit, Ollama runs on CPU (slower but works)

Edge Docker Compose (Raspberry Pi 5)

The edge Compose file for a Raspberry Pi 5 uses ARM-compatible images, omits Whisper and Tesseract, uses a small embedding model, and configures Qdrant for low memory. All services must use ARM64 images.

# Edge (Raspberry Pi 5): minimal, ARM64 images only
version: '3.8'

services:
  qdrant:
    image: qdrant/qdrant:v1.12.4          # ARM64 multi-arch
    ports:
      - "6333:6333"
    volumes:
      - qdrant_data:/qdrant/storage
    environment:
      - QDRANT__STORAGE__OPTIMIZERS__INDEXING_THRESHOLD=20000
      - QDRANT__STORAGE__QUANTIZATION__SCALAR=true

  postgres:
    image: postgres:16-alpine             # ARM64
    ports:
      - "5432:5432"
    volumes:
      - pg_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=rag
      - POSTGRES_USER=rag
      - POSTGRES_PASSWORD=localrag

  ollama:
    image: ollama/ollama:0.5.4            # ARM64
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    # No GPU block; CPU inference only
    # Pre-pull: ollama pull llama3.2:3b-q4_K_M

  embedding:
    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
    ports:
      - "8081:80"
    # Uses all-MiniLM-L6-v2 (384 dim, ~90MB)

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  orchestrator:
    image: sovereign-orchestrator:arm64
    ports:
      - "8000:8000"
    depends_on:
      - qdrant
      - postgres
      - ollama
      - embedding
      - redis

  webui:
    image: sovereign-webui:arm64
    ports:
      - "8080:80"
    depends_on:
      - orchestrator

  # No Prometheus/Grafana/cAdvisor on Pi 5 (save resources)
  # No Tesseract/Whisper (too heavy for 8GB RAM)

volumes:
  qdrant_data:
  pg_data:
  ollama_data:

Data Transfer for Air-Gapped Systems

The air-gapped pattern's defining constraint is that data moves in and out only via physical media. This makes data transfer a formalized, audited process -- not an afterthought. There are two things that need to enter the air-gapped system: documents for ingestion, and software/model updates. Both follow a similar procedure: prepare on a connected machine, transfer to media, scan and verify at a transfer station, carry media to the air-gapped server, ingest.

Document Ingestion via USB

Documents are prepared on a connected workstation (the "preparation machine"). The documents are written to a USB 3.2 external SSD. Before the USB drive is connected to the air-gapped server, it passes through a dedicated transfer station -- a separate machine that runs antivirus scanning (ClamAV with up-to-date signatures), checks file types against an allowlist, and verifies file integrity hashes. Only after the transfer station reports clean do the documents get ingested.

# Transfer station: scan and verify before air-gapped ingestion
# Run on the transfer station (connected machine with ClamAV)

#!/bin/bash
# Usage: ./transfer_check.sh /media/usb/documents

USB_DIR="$1"
QUARANTINE=/tmp/quarantine
LOG=/var/log/transfer_station.log

# 1. Update ClamAV signatures (transfer station has internet)
freshclam --quiet

# 2. Scan all files on the USB drive
echo "[$(date)] Scanning $USB_DIR..." | tee -a $LOG
clamscan --infected --recursive --move=$QUARANTINE $USB_DIR 2>&1 | tee -a $LOG

if [ $? -ne 0 ]; then
    echo "[$(date)] MALWARE DETECTED. Transfer ABORTED." | tee -a $LOG
    exit 1
fi

# 3. Check file types against allowlist
echo "[$(date)] Checking file types..." | tee -a $LOG
ALLOWED="pdf docx xlsx pptx txt md csv html epub tiff png jpg"
find $USB_DIR -type f | while read f; do
    ext=$(echo "$f" | awk -F. '{print tolower($NF)}')
    if ! echo "$ALLOWED" | grep -qw "$ext"; then
        echo "[$(date)] DISALLOWED: $f (.$ext)" | tee -a $LOG
        exit 2
    fi
done

# 4. Generate integrity manifest
echo "[$(date)] Generating SHA256 manifest..." | tee -a $LOG
cd $USB_DIR
find . -type f -exec sha256sum {} + > manifest.sha256

echo "[$(date)] Transfer approved. Ready for air-gapped ingestion." | tee -a $LOG
echo "Verify manifest on air-gapped side: sha256sum -c manifest.sha256"

Model Updates for Air-Gapped Systems

Model updates (new embedding models, new LLM versions) follow a stricter procedure than document ingestion. Models are downloaded on a connected machine, verified against published signatures, written to external media, scanned, and loaded onto the air-gapped server. Every model file has a known hash that is published by the model author. The transfer station verifies the hash before the model is approved for transfer. On the air-gapped server, the hash is verified again after the file is copied from the media, before it is loaded into Ollama or the embedding service.

# Model update transfer: download, verify, package for air-gapped
# Run on connected machine

#!/bin/bash
MODEL="$1"           # e.g., "llama3.1:8b"
TRANSFER_DIR=/media/usb/models
EXPECTED_HASH="$2"     # Published SHA256 from model author

# 1. Pull model on connected machine
ollama pull $MODEL

# 2. Export model to file
ollama show $MODEL --modelfile > $TRANSFER_DIR/$MODEL.modelfile
cp ~/.ollama/models/manifests/* $TRANSFER_DIR/manifests/
cp -r ~/.ollama/models/blobs/* $TRANSFER_DIR/blobs/

# 3. Verify hash
cd $TRANSFER_DIR
for f in blobs/*; do
    actual=$(sha256sum "$f" | awk '{print $1}')
    if [ "$actual" != "$EXPECTED_HASH" ]; then
        echo "HASH MISMATCH for $f. ABORT."
        exit 1
    fi
done

# 4. Sign the transfer package (GPG)
gpg --sign --detach $TRANSFER_DIR/manifests/registry.ollama.ai/library/$MODEL/latest

echo "Model $MODEL verified and signed. Ready for transfer."

# === On the air-gapped server ===
# 5. Verify GPG signature
gpg --verify $TRANSFER_DIR/manifests/*.sig

# 6. Re-verify hashes
cd $TRANSFER_DIR
for f in blobs/*; do
    actual=$(sha256sum "$f" | awk '{print $1}')
    echo "$f: $actual"
    # Compare against published hashes (printed on paper, not digital)
done

# 7. Load into Ollama
cp $TRANSFER_DIR/manifests/* ~/.ollama/models/manifests/
cp $TRANSFER_DIR/blobs/* ~/.ollama/models/blobs/
ollama list    # Verify model appears

⚠️ The Two-Person Rule For high-security air-gapped deployments, model and document transfers should follow a two-person rule: two authorized individuals must be present for the transfer, and both must verify the hashes and sign the transfer log. This prevents a single individual from introducing a malicious model or document. The transfer log (who, what, when, hashes verified) becomes part of the audit trail.

Backup Strategies Per Pattern

Every deployment pattern needs a backup strategy. The components that must be backed up are the same across all patterns: Qdrant vector snapshots, Postgres dumps, the document file store, and Ollama model files. What differs is where backups are stored and how they get there. The table below summarizes the backup strategy for each pattern.

Pattern	Primary Backup Target	Secondary Backup	Frequency	Retention
Air-Gapped	External USB drive (rotating set of 3)	Second external drive stored offsite	Weekly (or after major ingestion)	4 weeks rotating, 3 monthly archives
Isolated LAN	NAS on RAG VLAN	External drive rotated weekly	Nightly (automated)	7 daily, 4 weekly, 12 monthly
Hybrid (local)	NAS or local disk	External drive weekly	Nightly (automated)	7 daily, 4 weekly
Local-First	External USB drive	Cloud backup of non-sensitive metadata (optional)	Nightly (automated)	7 daily, 4 weekly
Edge	External USB drive or SD card	None (single device)	Manual, weekly	2 snapshots

Backup Script (All Patterns)

The same backup script works for all patterns; only the target directory changes. For air-gapped, the target is a mounted USB drive. For isolated LAN, it is a NAS share. For local-first, it is an external drive. The script creates three artifacts: a Qdrant snapshot, a Postgres dump, and a tarball of the document file store.

# Sovereign RAG backup script
# Usage: ./backup.sh /backup/target

#!/bin/bash
BACKUP_DIR="$1"/rag-backup-$(date +%Y%m%d-%H%M%S)
mkdir -p $BACKUP_DIR

# 1. Qdrant snapshot (via API)
curl -X PUT http://localhost:6333/collections/documents/snapshots
sleep 5  # Wait for snapshot to complete
curl http://localhost:6333/collections/documents/snapshots | python3 -c "
import json, sys, urllib.request
snapshots = json.load(sys.stdin)['result']['snapshots']
for s in snapshots:
    urllib.request.urlretrieve(f'http://localhost:6333/collections/documents/snapshots/{s[\"name\"]}', f'$BACKUP_DIR/qdrant_{s[\"name\"]}')
    print(f'Downloaded: {s[\"name\"]}')
"

# 2. Postgres dump
docker exec rag-postgres pg_dump -U rag -Fc rag > $BACKUP_DIR/postgres.dump

# 3. Document file store
tar czf $BACKUP_DIR/documents.tar.gz /data/rag/documents/

# 4. Generate manifest and sign
cd $BACKUP_DIR
find . -type f -exec sha256sum {} + > backup.manifest
gpg --sign --detach backup.manifest

# 5. Cleanup old backups (keep 7 daily)
find "$1"/rag-backup-* -maxdepth 0 -type d -mtime +7 -exec rm -rf {} \;

echo "Backup complete: $BACKUP_DIR"
echo "Verify: sha256sum -c backup.manifest"

⚠️ Test Restores -- Always An untested backup is not a backup. For each pattern, schedule a monthly restore test: load the Qdrant snapshot into a fresh Qdrant instance, restore the Postgres dump, extract the documents tarball. Run test queries and verify that answers and citations are correct. For air-gapped systems, this means dedicating a test server (also air-gapped) for restore verification. If you cannot verify a restore, you do not have a backup -- you have a hope.

Monitoring Per Pattern

Monitoring a sovereign RAG system means tracking the health of every service, the performance of queries, the state of the GPU, and the integrity of the data. The tools differ by pattern -- air-gapped systems cannot send alerts via email or Slack, while local-first systems can optionally forward metrics to a cloud monitoring service. The table below maps monitoring capabilities to patterns.

Capability	Air-Gapped	Isolated LAN	Local-First	Edge
Prometheus metrics	Yes (local)	Yes	Yes	Optional
Grafana dashboards	Yes (local)	Yes	Yes	Optional
cAdvisor (container metrics)	Yes	Yes	Yes	Optional
Syslog (local)	Yes (required)	Yes	Yes	Yes
Email/Slack alerts	No (no internet)	No (no internet)	Yes (optional)	Yes (optional)
Cloud monitoring (e.g., Grafana Cloud)	No	No	Optional	No
SNMP traps to NOC	No	Yes (if NOC exists)	No	No
Physical dashboard (screen in server room)	Yes (recommended)	Optional	No	No

Air-Gapped Monitoring

Without internet, alerting must be local. Prometheus collects metrics from all services. Grafana displays dashboards on a monitor directly connected to the server (or on a nearby machine in the same room). Alerts are written to syslog and displayed on a local status page. For critical failures (GPU OOM, Qdrant down, disk full), an audible alarm (a simple script that plays a sound through the server's speakers) can be configured. A physical dashboard -- a monitor mounted in the server room showing the Grafana overview -- is the air-gapped equivalent of a NOC display.

# Air-gapped alert: local audible alarm on critical failure
import subprocess
import requests
import time

ALERT_SOUND = '/usr/share/sounds/alert.wav'
CHECK_INTERVAL = 60  # seconds

def check_health():
    checks = {
        'qdrant': 'http://localhost:6333/healthz',
        'postgres': 'http://localhost:8000/health/postgres',
        'ollama': 'http://localhost:11434/api/tags',
        'embedding': 'http://localhost:8081/health',
    }
    failures = []
    for name, url in checks.items():
        try:
            r = requests.get(url, timeout=5)
            if r.status_code != 200:
                failures.append(name)
        except:
            failures.append(name)
    return failures

while True:
    failures = check_health()
    if failures:
        # Log to syslog
        subprocess.run(['logger', -p 'user.err',
                       f'RAG ALERT: {", ".join(failures)} down'])
        # Play alarm sound (repeated)
        for _ in range(3):
            subprocess.run(['aplay', ALERT_SOUND])
            time.sleep(1)
    time.sleep(CHECK_INTERVAL)

Isolated LAN Monitoring

Prometheus and Grafana run on the RAG server. Grafana is accessible from the LAN (and via VPN for remote users). Alert rules in Prometheus trigger on service failures, high GPU utilization, low disk space, and query latency spikes. Alerts are sent to a local syslog server and optionally to an internal messaging system (Mattermost, Rocket.Chat) running on the LAN. If the organization has a NOC (Network Operations Center), SNMP traps can be forwarded to the NOC's monitoring system.

Local-First Monitoring

Same Prometheus and Grafana stack as isolated LAN. The difference is alerting: because the local-first server has internet access (restricted, but present), alerts can be sent via email (SMTP to an external relay), to a Slack webhook, or to a Telegram bot. Optionally, metrics can be forwarded to Grafana Cloud or another cloud monitoring service for remote visibility. This is the only pattern where cloud-based monitoring adds value without compromising sovereignty -- the metrics themselves (CPU, disk, latency) are not sensitive.

Edge Monitoring

On a Raspberry Pi 5, running Prometheus and Grafana is possible but consumes scarce resources. The pragmatic approach is to run a lightweight health-check script (similar to the air-gapped audible alarm, above) that logs to syslog and optionally writes a status file that the web UI can display. For a laptop or mini PC with more resources, the full Prometheus + Grafana stack can run alongside the RAG services.

💡 What to Monitor (All Patterns): Service health: HTTP health endpoint of every service (Qdrant, Postgres, Ollama, embedding, re-ranker, Tika, orchestrator, web UI). GPU: Utilization, memory usage, temperature (via nvidia-smi or DCGM exporter). Disk: Free space on data volumes, I/O wait. Query performance: p50 and p95 query latency, embedding time, retrieval time, LLM generation time. Ingestion queue: Queue depth (Redis LLEN), throughput. Error rate: HTTP 5xx responses, Ollama generation failures, parse failures.

Decision Guide: Which Pattern to Choose

The decision of which deployment pattern to use is driven by five factors: data sensitivity, number of users, remote access needs, budget, and maintenance capacity. The matrix below maps these factors to patterns. Start by answering the five questions, then use the decision flow to arrive at a recommended pattern.

Question	Your Answer	Implication
How sensitive is your data?	Classified / existential if leaked	Air-Gapped
	Regulated (HIPAA, attorney privilege, financial)	Isolated LAN
	Sensitive but not regulated	Local-First
	Mixed (some sensitive, some public)	Hybrid
	Personal / non-sensitive	Edge or Local-First
How many users need access?	1 (just me)	Edge or Local-First
	2-10 (small team)	Local-First or Isolated LAN
	10-50 (department)	Isolated LAN
Do users need remote access?	No (all on-site)	Any pattern works
	Yes (occasionally)	Isolated LAN + VPN, or Local-First + WireGuard
	Yes (frequently)	Isolated LAN + VPN
What is your hardware budget?	< $200	Edge (Raspberry Pi)
	$500-$1,000	Local-First (mini PC)
	$2,000-$5,000	Local-First (server with GPU)
	$5,000-$15,000	Isolated LAN (server + networking)
	$15,000+	Air-Gapped (server room)
What is your maintenance capacity?	Minimal (I want it to just work)	Local-First or Edge
	Moderate (weekly check-ins)	Isolated LAN
	Dedicated staff + procedures	Air-Gapped

The decision flow below simplifies the matrix into a sequential process. Answer each question in order. If you reach a pattern, that is your starting recommendation. You can always adjust based on additional constraints.

DECISION FLOW: Q1: Is your data classified or existential if leaked? | +-- YES --> AIR-GAPPED (proceed to Q5 for confirmation) | +-- NO --> Q2 Q2: Is your data regulated (HIPAA, attorney privilege, GLBA)? | +-- YES --> Q3 (likely Isolated LAN) | +-- NO --> Q4 Q3: Do you have > 1 user or need remote access? | +-- YES --> ISOLATED LAN (with VPN) | +-- NO --> LOCAL-FIRST (single user, regulated data) Q4: Do you have a clear split between sensitive and public docs? | +-- YES, and public corpus is large --> HYBRID | +-- NO --> Q5 Q5: How many users, and what budget? | +-- 1 user, < $200 --> EDGE (Raspberry Pi) +-- 1-5 users, $500+ --> LOCAL-FIRST (mini PC or server) +-- 5+ users, $5000+ --> ISOLATED LAN

Migration Paths

Deployment patterns are not permanent. Organizations start simple and move toward more sovereign patterns as their data sensitivity or threat model evolves. They also start strict and loosen as they gain confidence in their security posture. Both directions are valid migrations. The key insight is that the RAG stack itself (Qdrant, Postgres, Ollama, embedding, re-ranker, orchestrator) does not change during migration. What changes is the network configuration, the hardware, and the operational procedures. This makes migration primarily a network and operations task, not a software rewrite.

Path 1: Starting Simple, Moving Up

The most common migration path is to start with local-first (the simplest pattern that provides sovereignty) and move toward more sovereign patterns as needs evolve. This path lets you get a working RAG system quickly, validate it with users, and then harden the deployment once the value is proven.

MIGRATION: LOCAL-FIRST → ISOLATED LAN → AIR-GAPPED Step 1: LOCAL-FIRST - Deploy on mini PC or server - Validate RAG quality with users - Monitor for 2-4 weeks - Identify which documents are ingested (data sensitivity audit) | | If data turns out to be regulated or sensitive: v Step 2: ISOLATED LAN - Move server to dedicated VLAN - Add firewall rules blocking outbound - Set up VPN for remote access - Add NAS for network backup - Migration takes 1-2 days (mostly network configuration) | | If threat model escalates or regulations tighten: v Step 3: AIR-GAPPED - Physically relocate server to secure room - Disconnect all network interfaces - Establish USB transfer procedures - Set up local NTP and HSM - Migration takes 1-2 weeks (physical move + procedure documentation)

Path 2: Starting Strict, Loosening

Less common but valid: an organization that deployed air-gapped may find that their threat model has changed (declassification, changed business model, new legal framework) and wants to move to a less restrictive pattern. This direction is easier than tightening -- you are adding capabilities, not removing them.

MIGRATION: AIR-GAPPED → ISOLATED LAN → LOCAL-FIRST Step 1: AIR-GAPPED - Server in secure room, no network - Data transfer via USB | | If threat model changes (declassification, policy shift): v Step 2: ISOLATED LAN - Connect server to isolated VLAN - Enable VPN for remote access - Data transfer now via network (still no internet) - USB procedures become backup-only - Migration takes 1 day (enable network, configure VLAN) | | If data is reclassified as non-regulated: v Step 3: LOCAL-FIRST - Allow controlled internet access (updates only) - Remove VPN (or keep for convenience) - Migration takes hours (firewall rule changes)

Path 3: Adding Hybrid

A local-first or isolated LAN deployment can be extended to hybrid by adding a cloud RAG half and a query router. This does not replace the local stack -- it augments it. The local stack continues to handle all sensitive queries. The cloud half handles public-knowledge queries. This migration is software-only (no hardware changes) and can be done in a day.

Path 4: Edge to Local-First

A user who started with an edge deployment (RAG on a Raspberry Pi or laptop) and wants to scale up to more documents or more users migrates to local-first by deploying the same Docker Compose stack on a more powerful machine. The Qdrant snapshots and Postgres dumps from the edge device can be loaded directly into the new server -- the data format is identical. This migration takes a few hours.

💡 Migration Principle: Because the RAG stack is the same set of Docker containers across all patterns, migration never involves rewriting the application. It involves changing where the containers run, how they are networked, and how data and updates get to them. This is the key design property that makes sovereign RAG migration practical: invest in containerization once, and you can redeploy across any pattern without software changes.

References and Further Reading

Deployment and Security

NIST SP 800-82 Rev. 3: Guide to Operational Technology (OT) Security -- Network isolation, air-gap security, and secure system architecture
NIST SP 800-53 Rev. 5: Security and Privacy Controls -- AC, SC, MP control families for isolated systems
NSA/CSS Manual for Air-Gapped Systems: nsa.gov -- Cross-domain solutions and data transfer procedures
WireGuard: wireguard.com -- Modern, fast VPN protocol for isolated LAN remote access
pfSense: pfsense.org -- Open-source firewall for VLAN isolation and outbound blocking
USBGuard: usbguard.github.io/usbguard -- USB device authorization for air-gapped systems
ClamAV: clamav.net -- Open-source antivirus for transfer station scanning
YubiHSM 2: yubico.com/products/yubihsm -- Hardware security module for key storage

RAG Infrastructure

Qdrant Vector Database: qdrant.tech/documentation -- Vector search, quantization, ARM64 support
Ollama: ollama.com -- Local LLM runtime, model management, GGUF quantization
Apache Tika: tika.apache.org -- Document parsing for 1000+ formats
Docker Compose: docs.docker.com/compose -- Multi-container orchestration for all patterns
Prometheus: prometheus.io -- Metrics collection for all deployment patterns
Grafana: grafana.com -- Dashboards for local and LAN monitoring

Avondale.AI Companion Guides

Building a Sovereign RAG Infrastructure Stack: avondale.ai/technical/rag/rag-infrastructure-stack.html
Sovereign Retrieval Philosophy: avondale.ai/technical/rag/rag-philosophy-sovereign-retrieval.html
RAG Explained -- Basic to Advanced: avondale.ai/technical/rag/rag-explained-basic-to-advanced.html
Embedding Models for Sovereign RAG: avondale.ai/technical/rag/embedding-models-sovereign-rag.html
Vector Database Comparison: avondale.ai/technical/rag/vector-database-comparison.html
Chunking Strategies for RAG: avondale.ai/technical/rag/chunking-strategies.html
Retrieval Methods: avondale.ai/technical/rag/retrieval-methods.html
Multimodal RAG: avondale.ai/technical/rag/multimodal-rag.html
Graph RAG and Knowledge Graphs: avondale.ai/technical/rag/graph-rag-knowledge-graphs.html
Medical RAG with DICOM and HIPAA: avondale.ai/technical/rag/medical-rag-dicom-hipaa.html
Business RAG Implementation: avondale.ai/technical/rag/business-rag-implementation.html
Home RAG Personal Assistant: avondale.ai/technical/rag/home-rag-personal-assistant.html
Local vs Cloud AI: avondale.ai/technical/local-vs-cloud-ai.html
HIPAA Compliance and AI: avondale.ai/technical/hipaa-compliance-ai.html
Model Quantization Guide: avondale.ai/technical/model-quantization-guide.html

Suggested Citation:
Sovereign RAG Deployment Patterns: From Air-Gapped to Edge. Avondale.AI Technical Documentation Library. Accessed June 2026. https://avondale.ai/technical/rag/sovereign-rag-deployment-patterns.html