AI-Powered Industrial Surveillance Platform

Unified Technical Blueprint — Part A: Sections 1-10

Cross-Reference Guide: This unified blueprint synthesizes six specialist design documents. For detailed specifications on any subsystem, refer to:

• architecture.md — Full architecture, scaling, failover, cost estimation
• video_ingestion.md — RTSP configuration, FFmpeg commands, edge gateway specs
• ai_vision.md — Model configurations, inference code, benchmarks
• database_schema.md — Complete DDL, triggers, views, RLS policies
• suspicious_activity.md — Detection algorithms, scoring engine pseudocode
• training_system.md — Training pipelines, quality gates, versioning logic

Table of Contents

• Section 1: Executive Summary
• Section 2: Kimi Swarm Team and Agent Responsibilities
• Section 3: Assumptions
• Section 4: Full Architecture
• Section 5: Data Flow from DVR to Cloud to Dashboard
• Section 6: Recommended Tech Stack
• Section 7: Database Schema
• Section 8: AI Model and Training Strategy
• Section 9: Suspicious Activity Night-Mode Design
• Section 10: Live Video Streaming Design

Section 1: Executive Summary

1.1 Project Objective

This blueprint defines the complete technical design for an AI-powered industrial surveillance platform that transforms a legacy CP PLUS 8-channel DVR system into a modern, intelligent security operations center. The platform processes real-time video from 8 camera channels, applies state-of-the-art computer vision and face recognition AI, detects suspicious activity during night hours, and provides a unified dashboard for security operators — all while maintaining the highest standards of reliability, security, and data privacy.

The system is designed around a cloud+edge hybrid architecture where all compute-intensive AI inference runs in the cloud (AWS Mumbai), while a local edge gateway handles stream ingestion, buffering, and site-local concerns. A WireGuard VPN tunnel protects all communication between edge and cloud, ensuring the DVR has zero public internet exposure.

1.2 Key Capabilities

1.3 Architecture Approach

The platform follows a cloud+edge+VPN hybrid pattern with five network security zones:

Cameras (8ch) --> DVR (local) --> Edge Gateway (local) --> WireGuard VPN --> AWS Cloud (EKS)
                                      |                        |
                                      | 2TB NVMe buffer         | Encrypted tunnel
                                      | 7-day ring buffer       | UDP 51820
                                      | FFmpeg ingestion        | ChaCha20-Poly1305

Key architectural decisions:

1.4 Target Environment

The platform targets a CP PLUS ORANGE CP-UVR-0801E1-CV2 DVR with the following characteristics:

Critical Design Impact: The DVR disk being full means the system cannot rely on DVR-side recording or playback features. All archival storage is managed by the edge gateway's 2TB NVMe buffer and cloud tiering.

1.5 Key Differentiators

1. AI Vibe Controls Instead of exposing complex threshold parameters to operators, the system provides three intuitive "vibe" presets — Relaxed, Balanced, and Strict — that internally map to optimized configurations for detection sensitivity and face match strictness. This innovation makes the system accessible to non-technical security staff while maintaining AI precision.

2. Safe Self-Learning Training System The platform captures operator corrections (confirmations, corrections, merges, rejections) and feeds them back into model improvement through a carefully designed three-mode learning pipeline: Manual Only, Suggested Learning (recommended), and Approved Auto-Update. A synchronous conflict detector blocks five types of label conflicts before they reach the training dataset, ensuring model integrity.

3. 24/7 Reliability with Graceful Degradation The system is architected around a single priority: video recording never stops. If the AI inference service fails, recording continues locally with queued catch-up processing on recovery. If the VPN tunnel fails, the edge gateway maintains 7 days of local buffer. If the cloud database fails, alerts accumulate in Kafka's durable log. Every failure mode has a defined degradation strategy.

4. 10-Module Night Surveillance The suspicious activity detection system goes beyond simple motion detection to provide comprehensive behavioral analysis through 10 specialized detection modules — from intrusion and loitering to abandoned objects and repeated re-entry patterns — all combined through a composite scoring engine with exponential time-decay.

1.6 Production Readiness Assessment

Section 2: Kimi Swarm Team and Agent Responsibilities

The unified blueprint was synthesized from the outputs of 11 specialist agents, each responsible for a specific domain of the platform design.

2.1 Agent Responsibility Matrix

2.2 Agent Interaction Flow

+-----------------------------------------------------------------------------+
|                         KIMI SWARM TEAM ORCHESTRATION                        |
+-----------------------------------------------------------------------------+
|                                                                              |
|   Requirements Analyst                                                       |
|        |                                                                     |
|        v                                                                     |
|   +---------+    +---------+    +---------+    +---------+                  |
|   | System  |<-->| Video   |<-->| AI      |<-->| Database|                  |
|   |Architect|    |Ingestion|    |Vision   |    |Architect|                  |
|   +---------+    +---------+    +---------+    +---------+                  |
|        ^                                              |                      |
|        |           +---------+    +---------+        |                      |
|        +---------->|Suspicious|<-->|Training |<-------+                      |
|                    |Activity  |    |System   |                               |
|                    |Designer  |    |Engineer |                               |
|                    +---------+    +---------+                               |
|                        |                                              |
|                        v                                              |
|                   +---------+    +---------+    +---------+           |
|                   |Frontend |    |DevOps   |    |Security |           |
|                   |Developer|    |Engineer |    |Architect|           |
|                   +---------+    +---------+    +---------+           |
|                        |                                              |
|                        v                                              |
|                   +---------------------+                             |
|                   | Technical Writer    |                             |
|                   | (Unified Blueprint) |                             |
|                   +---------------------+                             |
|                                                                              |
+-----------------------------------------------------------------------------+

2.3 Cross-Agent Design Consistency

The following cross-cutting concerns were harmonized across all agent outputs during synthesis:

Section 3: Assumptions

All assumptions made across the specialist designs are consolidated below. These should be validated before implementation begins.

3.1 Network and Hardware Assumptions

3.2 DVR Capabilities Assumptions

3.3 Environmental Assumptions

3.4 Operational Assumptions

3.5 Security Assumptions

3.6 AI Performance Assumptions

Section 4: Full Architecture

4.1 High-Level System Architecture

The platform employs a cloud+edge hybrid architecture with five network security zones. Video streams are ingested at the edge, processed by AI in the cloud, and presented through a web-based dashboard. A WireGuard VPN tunnel provides encrypted, zero-exposure connectivity between edge and cloud.

+=============================================================================+
|                         CLOUD+EDGE+VPN ARCHITECTURE                          |
+=============================================================================+
|                                                                              |
|   ZONE 0: INTERNET (UNTRUSTED)                                               |
|   +---------------------+                                                    |
|   |  Users / Browsers   |                                                    |
|   |  HTTPS :443         |                                                    |
|   +----------+----------+                                                    |
|              |                                                               |
|              v                                                               |
|   ZONE 1: AWS VPC EDGE (DEMILITARIZED)                                       |
|   +--------------------------------------------------------------+          |
|   |  AWS ALB (:443) + WAF v2 + Rate Limit + Geo-Restriction      |          |
|   |       |                                                      |          |
|   |       v                                                      |          |
|   |  Traefik Ingress Controller (:8443)                          |          |
|   |  - Route: /api/*  -> Backend Service                         |          |
|   |  - Route: /ws/*   -> WebSocket Handler                       |          |
|   |  - Route: /       -> Next.js Web App                         |          |
|   |  - TLS: Let's Encrypt auto certificates                      |          |
|   +--------------------------------------------------------------+          |
|              |                                                               |
|              v                                                               |
|   ZONE 2: AWS VPC APPLICATION (TRUSTED)                                      |
|   +--------------------------------------------------------------+          |
|   |  +-------------+  +-------------+  +---------------------+   |          |
|   |  | Stream      |  | AI Inference|  | Suspicious Activity |   |          |
|   |  | Ingestion   |  | Service     |  | Service (Night Mode)|   |          |
|   |  | (Go/FFmpeg) |  | (Triton)    |  | (Go/Python)         |   |          |
|   |  | :8081       |  | :8001 gRPC  |  | :8083               |   |          |
|   |  +-------------+  +-------------+  +---------------------+   |          |
|   |  +-------------+  +-------------+  +---------------------+   |          |
|   |  | Backend API |  | Training    |  | Notification        |   |          |
|   |  | (Go/Gin)    |  | Service     |  | Service             |   |          |
|   |  | :8080       |  | (PyTorch)   |  | (Go)                |   |          |
|   |  +-------------+  +-------------+  +---------------------+   |          |
|   |  +--------------------+                                      |          |
|   |  | Web Frontend       |  HLS Playback Service               |          |
|   |  | (Next.js 14 :3000) |  (Go :8085)                         |          |
|   |  +--------------------+                                      |          |
|   +--------------------------------------------------------------+          |
|              |                                                               |
|              v                                                               |
|   ZONE 3: AWS VPC DATA (HIGHLY RESTRICTED)                                   |
|   +--------------------------------------------------------------+          |
|   |  +-------------+  +-------------+  +-------------+           |          |
|   |  | PostgreSQL  |  | Redis       |  | Kafka       |           |          |
|   |  | 16 (RDS)    |  | 7 Cluster   |  | (MSK)       |           |          |
|   |  | :5432       |  | :6379       |  | :9092       |           |          |
|   |  | pgvector    |  | Pub/Sub     |  | 3 brokers   |           |          |
|   |  | HNSW index  |  | Streams     |  | 3 AZs       |           |          |
|   |  +-------------+  +-------------+  +-------------+           |          |
|   |  +-------------+  +-----------------------------------+      |          |
|   |  | MinIO       |  | S3 (Cold Archive)                 |      |          |
|   |  | (S3-compat) |  | - Standard (30d)                  |      |          |
|   |  | :9000       |  | - IA (31-90d)                     |      |          |
|   |  | 10 TB       |  | - Glacier Deep Archive (90d+)     |      |          |
|   |  +-------------+  +-----------------------------------+      |          |
|   +--------------------------------------------------------------+          |
|              |                                                               |
|              | WireGuard VPN Tunnel (UDP 51820)                                |
|              | ChaCha20-Poly1305 encryption                                    |
|              | Cloud peer: 10.200.0.1/32 <-> Edge peer: 10.200.0.2/32         |
|              v                                                               |
|   ZONE 4: EDGE NETWORK (PHYSICALLY ISOLATED)                                 |
|   +--------------------------------------------------------------+          |
|   |  +--------------------------------------------------------+  |          |
|   |  |              EDGE GATEWAY (Intel NUC)                  |  |          |
|   |  |  Ubuntu 22.04 LTS | K3s v1.28+ | 2TB NVMe             |  |          |
|   |  |                                                          |  |          |
|   |  |  +-----------------+  +-----------------+                |  |          |
|   |  |  | Stream Manager  |  | HLS Segmenter   |                |  |          |
|   |  |  | (Python/asyncio)|  | (FFmpeg/nginx)  |                |  |          |
|   |  |  | 8x RTSP feeds   |  | 2s segments     |                |  |          |
|   |  |  +-----------------+  +-----------------+                |  |          |
|   |  |  +-----------------+  +-----------------+                |  |          |
|   |  |  | Frame Extractor |  | Buffer Manager  |                |  |          |
|   |  |  | (AI decimation) |  | (20GB ring buf) |                |  |          |
|   |  |  +-----------------+  +-----------------+                |  |          |
|   |  |  +--------------------------------------------------+    |  |          |
|   |  |  | VPN Client (WireGuard)  |  Health Monitor         |    |  |          |
|   |  |  +--------------------------------------------------+    |  |          |
|   |  +--------------------------------------------------------+  |          |
|   |                            |                                             |
|   |   Local Network (192.168.29.0/24)                                       |
|   |   +------------------+    +------------------+                           |
|   |   | CP PLUS DVR      |    | Local Monitor    |                           |
|   |   | 192.168.29.200   |    | 192.168.29.10    |                           |
|   |   | 8ch | RTSP :554  |    | (optional)       |                           |
|   |   +------------------+    +------------------+                           |
|   |   CH1 CH2 CH3 CH4 CH5 CH6 CH7 CH8                                      |
|   +--------------------------------------------------------------+          |
|                                                                              |
+=============================================================================+

4.2 Service Interaction Diagram

+-----------------------------------------------------------------------------+
|                           SERVICE INTERACTIONS                               |
+-----------------------------------------------------------------------------+
|                                                                              |
|   INTERNET USERS                                                             |
|        |                                                                     |
|        | HTTPS :443                                                          |
|        v                                                                     |
|   +---------+      +----------+      +----------+                           |
|   | AWS ALB |----->| Traefik  |----->| Next.js  |  Web Frontend             |
|   | +WAF    |      | Ingress  |      | (SSR)    |  Dashboard                |
|   +---------+      +----------+      +----+-----+                           |
|                                             |                                |
|                        +--------------------+--------------------+           |
|                        |                    |                    |           |
|                        v                    v                    v           |
|                   +---------+       +------------+      +----------+       |
|                   |Backend  |       | WebSocket  |      | HLS      |       |
|                   |API (Go) |       | Handler    |      | Playback |       |
|                   |:8080    |       | /ws/alerts |      | Service  |       |
|                   +----+----+       +------------+      +----+-----+       |
|                        |                                               |
|                        | gRPC :50051                                    |
|                        v                                               |
|   +---------+    +------------+    +----------+    +----------+       |
|   | Stream  |    | AI         |    |Suspicious|    |Training  |       |
|   |Ingestion|<-->| Inference  |<-->| Activity |    |Service   |       |
|   |(Go)     |    |(Triton)    |    |(Night)   |    |(PyTorch) |       |
|   +----+----+    +------+-----+    +----+-----+    +----+-----+       |
|        |                |               |               |             |
|        v                v               v               v             |
|   +---------------------------------------------------------------+   |
|   |                        KAFKA (MSK)                            |   |
|   |  streams.raw (8 parts)  ai.detections (16 parts)             |   |
|   |  alerts.critical (4 parts)  training.data (30-day ret.)      |   |
|   |  notifications.*  system.metrics (7-day ret.)                |   |
|   +---------------------------------------------------------------+   |
|        |                |               |               |             |
|        v                v               v               v             |
|   +---------+    +------------+    +----------+    +----------+       |
|   |PostgreSQL|   | Redis      |    | MinIO    |    | MLflow   |       |
|   |16 +pgvec |   |7 Cluster   |    |S3-compat |    | Model    |       |
|   |:5432     |   |:6379       |    |:9000     |    | Registry |       |
|   +---------+    +------------+    +----------+    +----------+       |
|                                                                              |
|   Edge Gateway: WireGuard peer at 10.200.0.2/32                            |
|   Stream Ingestion pulls frames via VPN -> sends to Kafka                   |
|                                                                              |
+-----------------------------------------------------------------------------+

4.3 Network Security Zones

Five security zones provide defense in depth, from the public internet to the physically isolated edge network.

+=============================================================================+
|                         NETWORK SECURITY ZONES                               |
+=============================================================================+
|                                                                              |
|  +---------------------------------------------------------------------+    |
|  |  ZONE 0: INTERNET (UNTRUSTED)                                        |    |
|  |  - Public users, any source IP                                        |    |
|  |  - AWS Shield Standard DDoS protection                               |    |
|  |  - Geo-restriction: allow specific countries only                    |    |
|  +---------------------------+-----------------------------------------+    |
|                              |                                               |
|                              | HTTPS :443                                    |
|                              v                                               |
|  +---------------------------------------------------------------------+    |
|  |  ZONE 1: AWS VPC EDGE (DEMILITARIZED)                                |    |
|  |  - ALB + WAF v2 (SQL injection, XSS, rate limiting rules)           |    |
|  |  - Traefik Ingress (:8443)                                          |    |
|  |  - Auth: JWT + RBAC, API keys for edge gateway                     |    |
|  |  - Public API endpoints ONLY                                        |    |
|  |  SG: alb-public-sg: 443 from 0.0.0.0/0                             |    |
|  |  SG: traefik-sg: 8443 from alb-sg ONLY                              |    |
|  +---------------------------+-----------------------------------------+    |
|                              |                                               |
|                              | Internal :8080-8090                         |
|                              v                                               |
|  +---------------------------------------------------------------------+    |
|  |  ZONE 2: AWS VPC APPLICATION (TRUSTED, ISOLATED)                     |    |
|  |  - Stream Ingestion, AI Inference, Suspicious Activity              |    |
|  |  - Training, Backend API, Notification Services                     |    |
|  |  - Pod Security: No root, read-only FS, no privilege escalation    |    |
|  |  - Network Policies: Ingress only from API GW namespace            |    |
|  |  SG: app-sg: 8080-8090 from traefik-sg ONLY                         |    |
|  +---------------------------+-----------------------------------------+    |
|                              |                                               |
|                              | Data Layer :5432, :6379, :9092, :9000       |
|                              v                                               |
|  +---------------------------------------------------------------------+    |
|  |  ZONE 3: AWS VPC DATA (HIGHLY RESTRICTED)                            |    |
|  |  - PostgreSQL (RDS), Redis (ElastiCache), Kafka (MSK)               |    |
|  |  - MinIO object storage, S3 cold archive                            |    |
|  |  - Security Groups: ONLY from app-sg                                |    |
|  |  - RDS: Encrypted at rest (AWS KMS), no public access              |    |
|  |  - S3: Bucket policy deny all except VPC endpoint                   |    |
|  +---------------------------+-----------------------------------------+    |
|                              |                                               |
|                              | WireGuard VPN (UDP 51820)                     |
|                              | ChaCha20-Poly1305                             |
|                              v                                               |
|  +---------------------------------------------------------------------+    |
|  |  ZONE 4: EDGE NETWORK (PHYSICALLY ISOLATED)                          |    |
|  |  - Edge Gateway (Intel NUC), K3s node                                |    |
|  |  - WireGuard peer, stream ingestion, local buffer                    |    |
|  |  - DVR (192.168.29.200): NO internet access, local ONLY             |    |
|  |  - Edge Firewall: ALLOW 192.168.29.0/24 -> DVR :554,:80           |    |
|  |                   ALLOW OUT 51820/udp -> Cloud VPN endpoint        |    |
|  |                   DENY ALL other incoming                           |    |
|  +---------------------------------------------------------------------+    |
|                                                                              |
+=============================================================================+

4.4 Service Descriptions

4.5 Physical Edge Gateway Specification

4.6 Cloud Infrastructure Specification

4.7 Scaling Approach

The system scales from the initial 8-camera deployment to 64+ cameras through well-defined phases:

+-----------------------------------------------------------------------------+
|                        CAMERA SCALING ROADMAP                                |
+-----------------------------------------------------------------------------+
|                                                                              |
|  CURRENT: 8 cameras (1 DVR)                                                  |
|  +-- Edge: Intel NUC i7, 32GB RAM                                           |
|  +-- Bandwidth: ~16 Mbps upstream (2 Mbps per H.264 stream)                 |
|  +-- Cloud AI: 1x T4 GPU (8 streams @ 1 fps, batch=8)                       |
|  +-- Kafka: 8 partitions (streams.raw)                                      |
|  +-- PostgreSQL: db.r6g.xlarge                                              |
|  +-- Monthly cost: ~$2,140                                                  |
|                                                                              |
|  PHASE 1: 16 cameras (2 DVRs / 2 sites)                                      |
|  +-- Edge: 2x Intel NUC (one per site)                                      |
|  +-- Bandwidth: ~32 Mbps                                                    |
|  +-- Cloud AI: 1x T4 GPU (batch=16, still sufficient)                       |
|  +-- Kafka: 16 partitions                                                   |
|  +-- Monthly cost: ~$3,200                                                  |
|                                                                              |
|  PHASE 2: 32 cameras (4 DVRs / 4 sites)                                      |
|  +-- Edge: 4x Intel NUC                                                     |
|  +-- VPN: Hub-spoke model (4 edge peers -> 1 cloud endpoint)                |
|  +-- Bandwidth: ~64 Mbps                                                    |
|  +-- Cloud AI: 2x T4 GPUs (HPA: 2-6 replicas)                               |
|  +-- Kafka: 32 partitions                                                   |
|  +-- PostgreSQL: db.r6g.2xlarge                                             |
|  +-- Monthly cost: ~$5,500                                                  |
|                                                                              |
|  PHASE 3: 64 cameras (8 DVRs / 8 sites)                                      |
|  +-- Edge: 8x Intel NUC (or Jetson Orin for edge AI pre-filter)              |
|  +-- Bandwidth: ~128 Mbps (dedicated circuit recommended)                   |
|  +-- Cloud AI: 4x T4 GPUs or 2x A10G (g5.2xlarge)                           |
|  +-- Kafka: 64 partitions, consider MSK multi-cluster                        |
|  +-- PostgreSQL: db.r6g.4xlarge + read replica                              |
|  +-- Monthly cost: ~$9,800                                                  |
|                                                                              |
+-----------------------------------------------------------------------------+

4.8 Failover and Reliability Design

The graceful degradation matrix defines behavior for every failure mode:

+=============================================================================+
|                     GRACEFUL DEGRADATION MATRIX                              |
+=============================================================================+
|                                                                              |
|  Failure Mode              | Degradation Strategy                            |
|  ------------------------- | ----------------------------------------------- |
|  AI Inference Service DOWN | Continue recording ALL video locally            |
|  (GPU failure, model crash)| Events stored as "unprocessed"                  |
|                            | No real-time alerts                             |
|                            | Queue frames for later batch processing         |
|                            | Dashboard shows "AI OFFLINE" banner             |
|                                                                              |
|  Kafka DOWN (MSK outage)   | Edge Gateway buffers locally (20GB ring buffer) |
|                            | Backpressure: reduce to key frames only (0.2fps)|
|                            | Auto-reconnect with 2x exponential backoff      |
|                            | Replay from local buffer when Kafka recovers    |
|                                                                              |
|  VPN Tunnel DOWN           | Full local operation mode                       |
|  (internet outage)         | All recording continues locally (7-day buffer)  |
|                            | Local alert buzzer/relay (configurable)         |
|                            | No cloud dashboard access                       |
|                            | Auto-sync when VPN recovers                     |
|                                                                              |
|  PostgreSQL DOWN (RDS)     | Alert queue builds in Kafka (durable log)       |
|                            | Events not lost (Kafka 7-day retention)         |
|                            | Read-only dashboard mode (Redis cache)          |
|                            | Alert on-call engineer                          |
|                                                                              |
|  Notification Service DOWN | Alerts accumulate in DB                         |
|                            | Retry with exponential backoff                  |
|                            | Dead letter after 24 hours                      |
|                            | Dashboard shows pending count                   |
|                                                                              |
|  Edge Gateway DOWN (power) | Cloud dashboard shows "SITE OFFLINE"            |
|                            | Last known recordings in cloud                  |
|                            | Alert sent immediately                          |
|                            | UPS: graceful shutdown, preserve data           |
|                                                                              |
+=============================================================================+

Priority Order (highest first):

1. Video recording NEVER STOPS (local edge priority)
2. Critical alerts ALWAYS FIRE (local buzzer + queued cloud alerts)
3. AI inference gracefully degrades to batch catch-up on recovery
4. Dashboard operates in read-only/cache mode during DB outage
5. Cloud sync resumes automatically when connectivity restored

Reliability Mechanisms:

Section 5: Data Flow from DVR to Cloud to Dashboard

This section traces the complete data journey from camera capture through AI processing to user presentation.

5.1 Overview: Seven Data Flows

+=============================================================================+
|                        SEVEN DATA FLOW PATHWAYS                              |
+=============================================================================+
|                                                                              |
|  Flow 1: Camera --> DVR --> Edge Gateway                                    |
|          [Analog/Digital] -> [H.264 Encode] -> [RTSP Server]                |
|                                                                              |
|  Flow 2: Edge Gateway --> VPN --> Cloud Kafka                               |
|          [FFmpeg ingest] -> [Frame extract] -> [Kafka Producer]             |
|                                                                              |
|  Flow 3: Stream Ingestion --> AI Inference                                  |
|          [Kafka Consumer] -> [GPU Batch] -> [Detection + Face Recog.]       |
|                                                                              |
|  Flow 4: AI Inference --> Events --> Database                               |
|          [Detection results] -> [Event enrich] -> [PostgreSQL]              |
|                                                                              |
|  Flow 5: Events --> Alerts --> Notifications                                |
|          [Scoring engine] -> [Alert create] -> [Multi-channel send]         |
|                                                                              |
|  Flow 6: Live Streams --> Browser Dashboard                                 |
|          [HLS segmenter] -> [Nginx relay] -> [HLS.js player]                |
|                                                                              |
|  Flow 7: Training Feedback Loop                                             |
|          [Operator review] -> [Conflict detect] -> [Model update]           |
|                                                                              |
+=============================================================================+

5.2 Flow 1: Camera to DVR to Edge Gateway

Path: Analog/Digital Camera -> DVR internal encoder -> DVR RTSP server -> Edge Gateway FFmpeg client

Protocol Stack:

FFmpeg RTSP Connection Command:

ffmpeg -hide_banner -loglevel warning \
    -rtsp_transport tcp \
    -stimeout 5000000 \
    -fflags +genpts+discardcorrupt+igndts+ignidx \
    -reorder_queue_size 64 \
    -i "rtsp://admin:password@192.168.29.200:554/cam/realmonitor?channel=1&subtype=0" \
    -c copy -f segment -segment_time 60 -reset_timestamps 1 \
    -strftime 1 "/data/buffer/ch1/%Y%m%d_%H%M%S.mkv"

Latency Budget:

5.3 Flow 2: Edge Gateway to VPN Tunnel to Cloud

Path: Edge Gateway FFmpeg -> Frame extraction -> JPEG encoding -> Kafka Producer -> WireGuard VPN -> Cloud MSK

Frame Processing Pipeline:

+------------+    +-------------+    +---------------+    +-------------+    +-----------+
| Raw RTSP   | -> | FFmpeg      | -> | Frame         | -> | JPEG        | -> | Kafka     |
| H.264      |    | Demux/Decode|    | Decimation    |    | Encoder     |    | Producer  |
| 25 FPS     |    |             |    | (1 fps)       |    | Quality 85  |    | (LZ4)     |
| 960x1080   |    |             |    | 640x640 crop  |    |             |    |           |
+------------+    +-------------+    +---------------+    +-------------+    +-----------+

FFmpeg Frame Extraction for AI:

ffmpeg -hide_banner -loglevel warning \
    -rtsp_transport tcp -stimeout 5000000 \
    -fflags +genpts+discardcorrupt -reorder_queue_size 64 \
    -i "rtsp://admin:password@192.168.29.200:554/cam/realmonitor?channel=1&subtype=0" \
    -vf "fps=1,scale=640:640:force_original_aspect_ratio=decrease,pad=640:640:(ow-iw)/2:(oh-ih)/2:black" \
    -q:v 5 -f image2pipe -vcodec mjpeg pipe:1

WireGuard VPN Tunnel Configuration:

VPN watchdog script runs every 30 seconds; restarts WireGuard on 3 consecutive ping failures.

Latency Budget:

5.4 Flow 3: Stream Ingestion to AI Inference

Path: Kafka streams.raw topic -> Stream Ingestion consumer -> Triton Inference Server -> Kafka ai.detections topic

Pipeline Architecture:

+------------+    +-------------------+    +------------------+    +-------------+
| streams.raw| -> | Stream Ingestion  | -> | NVIDIA Triton    | -> | ai.detections |
| (8 parts)  |    | (Go consumer)     |    | (GPU inference)  |    | (16 parts)    |
| JPEG frames|    | Batch aggregator  |    | gRPC :8001       |    | Detection     |
| + metadata |    | (batch=8, timeout)|    | Dynamic batching |    | + embeddings  |
+------------+    +-------------------+    +------------------+    +-------------+

Triton Model Configuration:

Total GPU memory: ~7.1 GB (fits in T4 16 GB with 8 streams)

Latency Budget:

5.5 Flow 4: AI Inference to Events to Database

Path: AI Detection results -> Event enricher -> PostgreSQL (multiple tables)

Data Transformation:

+------------+    +-------------------+    +---------------------+    +------------+
| Detection  | -> | Event Enricher    | -> | PostgreSQL Writer   | -> | events     |
| results    |    | - Add camera_id   |    | - UPSERT person     |    | persons    |
| (raw)      |    | - Match person    |    | - INSERT event      |    | embeddings |
|            |    | - Check whitelist |    | - INSERT embedding  |    | face_crops |
+------------+    +-------------------+    +---------------------+    +------------+

Database Write Operations per Detection:

5.6 Flow 5: Events to Alerts to Notifications

Path: AI events -> Suspicious Activity scoring engine -> Alert creation -> Notification dispatch

Scoring and Escalation:

+------------+    +-------------------+    +------------------+    +-------------+
| AI events  | -> | Suspicious Activity| -> | Alert Manager    | -> | Notification |
| (persons,  |    | Scoring Engine     |    | - Deduplicate    |    | Service      |
|  faces)    |    | - 10 modules       |    | - Rate limit     |    | - Telegram   |
|            |    | - Composite score  |    | - Suppress dup   |    | - WhatsApp   |
|            |    | - Time decay       |    | - Escalation     |    | - Email      |
+------------+    +-------------------+    +------------------+    +-------------+

Alert Escalation Matrix:

5.7 Flow 6: Live Streams to Browser Dashboard

Path: DVR RTSP -> Edge Gateway FFmpeg -> HLS segmenter -> Nginx -> CDN -> Browser HLS.js

+--------+    +---------------+    +---------------+    +---------+    +----------+
| DVR    | -> | Edge Gateway  | -> | HLS Segmenter | -> | Nginx   | -> | Browser  |
| RTSP   |    | FFmpeg        |    | (2s segments) |    | (relay) |    | HLS.js   |
| 25 FPS |    | -copyts       |    | H.264 + AAC   |    | HTTPS   |    | Video tag|
+--------+    +---------------+    +---------------+    +---------+    +----------+

HLS Configuration:

Latency:

5.8 Flow 7: Training Feedback Loop

Path: Operator review actions -> Conflict detection -> Training dataset -> Model training -> Quality gates -> Deployment

+------------+    +------------------+    +----------------+    +-------------+    +-----------+
| Operator   | -> | Conflict         | -> | Training       | -> | Quality     | -> | Deployment |
| Review     |    | Detection        |    | Dataset        |    | Gates       |    | (A/B test) |
| (confirm,  |    | (5 types)        |    | - Curate       |    | - Precision |    |            |
|  correct,  |    | - Block conflicts|    | - Label        |    |   >= 0.97   |    |            |
|  merge,    |    | - Queue safe     |    | - Augment      |    | - Recall    |    |            |
|  reject)   |    |   additions      |    | - Version      |    |   >= 0.95   |    |            |
+------------+    +------------------+    +----------------+    +-------------+    +-----------+

Training Data Flow:

Section 6: Recommended Tech Stack

6.1 Technology Selection Matrix

6.2 Hardware Requirements

Edge Gateway (Per Site)

Cloud GPU Nodes (AI Inference)

6.3 Software Versions Summary

6.4 Port Reference

Section 7: Database Schema

7.1 Schema Overview

The database is designed around a relational core (PostgreSQL 16) with pgvector extension for 512-dimensional face embedding storage and similarity search. The schema consists of 29 tables, 4 views, and 8 trigger functions, organized into 10 logical domains.

Schema Philosophy:

• Strict normalization for reference data (cameras, persons, rules) to ensure data integrity
• JSONB flexibility for event metadata and configuration to accommodate evolving AI outputs
• Partitioning on all high-volume time-series tables for query performance and lifecycle management
• pgvector HNSW indexing for sub-10ms face similarity search at scale
• Row-level security (RLS) for multi-tenant site isolation
• AES-256 encryption for all stored credentials (DVR passwords, API tokens)

7.2 Entity Relationship Overview

+=============================================================================+
|                    ENTITY RELATIONSHIP DIAGRAM                               |
+=============================================================================+
|                                                                              |
|   SITE (1) --------------------< (N) DVR                                     |
|    |                              |                                          |
|    |                              | (1)                                      |
|    |                              v                                          |
|    |                           CAMERA (N) <------------------< (N) ALERT_RULE|
|    |                              |                              |           |
|    |                              | (N)                            | (1)      |
|    |                              v                              v           |
|    |   +---------------------------------------------------------+           |
|    |   | EVENT (N) -->--(1) PERSON (1)--< (N) FACE_EMBEDDING               |
|    |   |   |                                                      |         |
|    |   |   | (N)                                                  | (N)     |
|    |   |   v                                                      v         |
|    |   | FACE_CROP (N)                                    PERSON_CLUSTER     |
|    |   |   |                                                                  |
|    |   |   | (N)                                                  +---------+|
|    |   |   v                                                      | Training||
|    |   | MEDIA_FILE (1) ----------------------------------------->| Dataset  ||
|    |   |                                                          |---------||
|    |   +--------------------------------------------------------->| Job      ||
|    |                                                              | Model    ||
|    |                              +---------+                     | Version  ||
|    |                              | Review  |                     +---------+|
|    |                              | Action  |                                |
|    |                              +---------+                                |
|    |                                    ^                                    |
|    |                                    | (N)                                |
|    +------------------------------------+                                    |
|   USER (N) -->--(N) ROLE_PERMISSION                                          |
|    |                                                                         |
|    | (1)                                                                     |
|    v                                                                         |
|   WATCHLIST (N) -->--(N) WATCHLIST_ENTRY                                     |
|                                                                              |
|   +---------+    +---------+    +---------+    +---------+                  |
|   | Telegram|    |WhatsApp |    | Email   |    |Webhook  |                  |
|   | Config  |    | Config  |    | Config  |    | Config  |                  |
|   +---------+    +---------+    +---------+    +---------+                  |
|        ^              ^             ^              ^                         |
|        |              |             |              |                         |
|        +--------------+-------------+--------------+                         |
|                         |                                                    |
|                   NOTIFICATION_CHANNEL                                         |
|                         |                                                    |
|                         | (1)                                                |
|                         v                                                    |
|                   NOTIFICATION_LOG                                             |
|                                                                              |
|   +---------+    +---------+    +---------+                                  |
|   | Audit   |    | System  |    | Device  |                                  |
|   | Log     |    | Health  |    | Connect.|                                  |
|   |(partitioned) |  Log    |    |  Log    |                                  |
|   +---------+    +---------+    +---------+                                  |
|                                                                              |
+=============================================================================+

7.3 Core Tables Summary

7.3.1 Site and Infrastructure Tables

7.3.2 AI Detection and Identity Tables

7.3.3 Alert and Notification Tables

7.3.4 Watchlist and Access Control Tables

7.3.5 Training and ML Pipeline Tables

7.3.6 Media and Storage Tables

7.3.7 Audit and Monitoring Tables (Partitioned)

7.4 Indexing Strategy

7.4.1 pgvector HNSW Index (Critical Path)

-- HNSW index for sub-10ms face similarity search
-- ef_search controls recall/speed tradeoff (higher = more accurate, slower)
CREATE INDEX idx_face_embeddings_hnsw
ON face_embeddings
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 128);

-- Query: Find top-K similar faces
SELECT person_id, 1 - (embedding <=> query_vector) AS similarity
FROM face_embeddings
WHERE is_primary = true
ORDER BY embedding <=> query_vector
LIMIT 5;

7.4.2 B-Tree Indexes (Standard Queries)

7.5 Partitioning Strategy

All high-volume time-series tables are partitioned monthly using pg_partman for automated partition management.

+-----------------------------------------------------------------------------+
|                    PARTITIONING ARCHITECTURE                                 |
+-----------------------------------------------------------------------------+
|                                                                              |
|   events (parent, empty)                                                     |
|   +-- events_y2024m01   (Jan 2024 data)                                     |
|   +-- events_y2024m02   (Feb 2024 data)                                     |
|   +-- events_y2024m03   (Mar 2024 data)                                     |
|   +-- events_y2024m04   (Apr 2024 data)                                     |
|   +-- events_y2024m05   (May 2024 data)  <-- Hot (in memory)               |
|   +-- events_default    (fallback)                                          |
|                                                                              |
|   Partition pruning: WHERE timestamp >= '2024-05-01'                        |
|                      -> Only scans events_y2024m05                           |
|                      -> ~30x faster for time-range queries                  |
|                                                                              |
|   Managed by: pg_partman extension                                          |
|   - Auto-create: 2 months ahead                                             |
|   - Auto-drop: After retention period (detach + archive)                    |
|                                                                              |
+-----------------------------------------------------------------------------+

Partitioned Tables:

7.6 Retention Policies

Automated Cleanup:

7.7 Security Considerations

7.7.1 Credential Encryption

All sensitive credentials stored with AES-256 encryption:

7.7.2 Row-Level Security (RLS)

For multi-site deployments, RLS policies enforce that users only see data for sites they have access to:

-- Enable RLS on critical tables
ALTER TABLE events ENABLE ROW LEVEL SECURITY;
ALTER TABLE persons ENABLE ROW LEVEL SECURITY;
ALTER TABLE alerts ENABLE ROW LEVEL SECURITY;

-- Policy: Users see only data from their assigned sites
CREATE POLICY site_isolation_events ON events
    USING (camera_id IN (
        SELECT c.id FROM cameras c
        JOIN dvrs d ON c.dvr_id = d.id
        JOIN site_users su ON d.site_id = su.site_id
        WHERE su.user_id = current_setting('app.current_user_id')::UUID
    ));

7.7.3 Access Control

7.7.4 Audit Trail

The audit_logs table (partitioned monthly) captures every significant action:

7.7.5 Backup Strategy

Reference: For complete DDL including all CREATE TABLE statements, triggers, views, and functions, see database_schema.md — Sections 2 through 15 contain the full schema definition with comments and constraints.

Section 8: AI Model and Training Strategy

8.1 AI Model Selection

The inference pipeline uses three complementary deep learning models — for human detection, face detection, and face recognition — all optimized with TensorRT for GPU inference. All models run on a single NVIDIA T4 GPU with dynamic batching.

8.1.1 Human Detection: YOLO11m

Export pipeline:

# PyTorch -> ONNX -> TensorRT Engine
yolo export model=yolo11m.pt format=onnx imgsz=640 half=True opset=17 simplify=True
trtexec --onnx=yolo11m.onnx --saveEngine=yolo11m.engine --fp16 \
  --minShapes=images:1x3x640x640 --optShapes=images:8x3x640x640 --maxShapes=images:16x3x640x640

8.1.2 Face Detection: SCRFD-500M-BNKPS

8.1.3 Face Recognition: ArcFace R100 (IR-SE100)

Published benchmarks on standard datasets:

8.2 Inference Pipeline Architecture

+=============================================================================+
|                    REAL-TIME INFERENCE PIPELINE                              |
+=============================================================================+
|                                                                              |
|  INPUT: RTSP Frame (640x640, 1 fps per stream)                              |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Frame Preprocessor| -> | YOLO11m Detector  | -> | Person Detection  |    |
|  | - Resize          |    | (TensorRT FP16)   |    | Results:          |    |
|  | - Normalize       |    | GPU: 12ms (P50)   |    | - bbox (x1,y1,x2, |    |
|  | - NCHW layout     |    | Batch: 1-16       |    |   y2)             |    |
|  +-------------------+    +-------------------+    | - confidence      |    |
|                                                     | - class (person)  |    |
|                                                     +---------+---------+    |
|                                                               |              |
|                                                               v              |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Face Crop Extract | <- | SCRFD-500M        | <- | Face Detection    |    |
|  | (ROI from person  |    | (TensorRT FP16)   |    | Results:          |    |
|  |  bounding box)    |    | GPU: 8ms (P50)    |    | - face bbox       |    |
|  |                   |    | Batch: per-face   |    | - 5 landmarks     |    |
|  +-------------------+    +-------------------+    | - confidence      |    |
|                                                     +---------+---------+    |
|                                                               |              |
|                                                               v              |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Face Alignment    | <- | ArcFace R100      | <- | Embedding Vector  |    |
|  | (5-point affine   |    | (TensorRT FP16)   |    | 512-D float32,   |    |
|  |  transform to     |    | GPU: 5ms (P50)    |    | L2-normalized     |    |
|  |  112x112)         |    | Batch: 1-64       |    |                   |    |
|  +-------------------+    +-------------------+    +---------+---------+    |
|                                                               |              |
|                                                               v              |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Face Matching     | <- | Person Tracking   | <- | Track-to-Person   |    |
|  | (cosine similarity|    | (ByteTrack)       |    | Association       |    |
|  |  vs. known DB)    |    | CPU: 2ms/frame    |    | - Match embedding |    |
|  +-------------------+    +-------------------+    |   to known persons  |    |
|       |  |  |                                      | - Create/update     |    |
|       |  |  |                                      |   track             |    |
|       v  v  v                                      +-------------------+    |
|  +-------------------+                                                        |
|  | Confidence Scorer |                                                        |
|  | (aggregate score  |                                                        |
|  |  for all detect)  |                                                        |
|  +-------------------+                                                        |
|       |                                                                       |
|       v                                                                       |
|  OUTPUT: DetectionEvent (JSON)                                               |
|  { person_id, track_id, confidence, bbox, face_crop,                         |
|    embedding, recognized_name?, quality_scores }                             |
|                                                                              |
+=============================================================================+

End-to-end latency budget per frame:

8.3 Face Recognition Matching Strategy

8.3.1 Known Person Matching

+-----------------------------------------------------------------------------+
|                    FACE RECOGNITION MATCHING FLOW                            |
+-----------------------------------------------------------------------------+
|                                                                              |
|  New Face Embedding (512-D)                                                  |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+                                                       |
|  | L2 Normalize      |  embedding = embedding / ||embedding||_2              |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+    +-------------------+                              |
|  | pgvector HNSW     | -> | Top-5 Candidates  |                              |
|  | Similarity Search |    | (cosine distance) |                              |
|  | ef_search=128     |    +-------------------+                              |
|  +-------------------+            |                                          |
|                                   v                                          |
|  +-------------------+    +-------------------+                              |
|  | Threshold Check   | <- | Best Match Score  |                              |
|  | (per AI Vibe)     |    +-------------------+                              |
|  +-------------------+            |                                          |
|       |                          |                                          |
|       +------------+-------------+                                          |
|                    |                                                        |
|         +----------+----------+                                             |
|         |                     |                                             |
|         v                     v                                             |
|    Above threshold      Below threshold                                     |
|    (Recognized)         (Unknown)                                           |
|         |                     |                                             |
|         v                     v                                             |
|  +------------+       +------------------+                                 |
|  | Assign to  |       | Check against    |                                 |
|  | known      |       | recent unknown   |                                 |
|  | person_id  |       | embeddings       |                                 |
|  | (with      |       | (5-min window)   |                                 |
|  | confidence)|       +--------+---------+                                 |
|  +------------+                |                                            |
|                                |                                            |
|                       +--------+--------+                                   |
|                       |                 |                                   |
|                       v                 v                                   |
|                  Similar unknown    No similar unknown                      |
|                  (same person)      (new unknown)                           |
|                       |                 |                                   |
|                       v                 v                                   |
|                  Reuse person_id   Create new                              |
|                  Update centroid   unknown person                           |
|                                    record                                   |
|                                                                              |
+-----------------------------------------------------------------------------+

8.3.2 AI Vibe Threshold Mapping

The AI Vibe system maps three intuitive presets to internal confidence thresholds:

Per-stream Vibe Selection:

• Vibe can be set per camera via dashboard
• Night mode automatically applies Strict vibe
• Alert-triggered cameras automatically upgrade to Strict for 5 minutes

8.4 Unknown Person Clustering Approach

Unknown persons (faces that don't match any known person above threshold) are automatically clustered to help operators identify recurring visitors.

8.4.1 Clustering Pipeline

+-----------------------------------------------------------------------------+
|                    UNKNOWN PERSON CLUSTERING PIPELINE                        |
+-----------------------------------------------------------------------------+
|                                                                              |
|  Unknown Face Embeddings (streaming)                                         |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+                                                       |
|  | Sliding Window    |  Keep last N embeddings in memory (configurable)     |
|  | Buffer (500)      |  + persistent storage for long-term clustering       |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+    +-------------------+                              |
|  | HDBSCAN Clustering| -> | Primary clusters  |  min_cluster_size=5        |
|  | (density-based)   |    | formed             |  min_samples=2             |
|  | metric=cosine     |    +-------------------+  eps=auto                   |
|  +-------------------+            |                                          |
|       | (fallback)                |                                          |
|       v                           v                                          |
|  +-------------------+    +-------------------+                              |
|  | DBSCAN Fallback   |    | Merge with        |  Check: temporal gap       |
|  | (if HDBSCAN fails |    | existing clusters |  < 30 days, cosine sim     |
|  |  to find structure|    | - centroid        |  > 0.85                    |
|  +-------------------+    |   distance        |                            |
|                           +-------------------+                            |
|                                   |                                          |
|                                   v                                          |
|                           +-------------------+                              |
|                           | Operator Review   |  Dashboard shows clusters   |
|                           | Queue             |  pending identification     |
|                           +-------------------+                              |
|                                                                              |
+-----------------------------------------------------------------------------+

8.4.2 Clustering Parameters

8.4.3 Clustering Quality Targets

8.5 Confidence Handling

8.5.1 Confidence Score Computation

Each detection event carries an aggregate confidence score computed from multiple signals:

confidence_aggregate = weighted_average(
    detection_confidence:    0.35 * yolo_confidence,
    face_detection_quality:  0.25 * scrfd_confidence,
    face_recognition_score:  0.25 * (1 - cosine_distance_to_match),
    face_quality_score:      0.15 * quality_composite
)

Where quality_composite = average(
    1.0 - blur_score,       # Sharpness (higher is better)
    1.0 - abs(pose_yaw)/90, # Frontal preference
    illumination_score,      # Well-lit face
    resolution_adequacy      # Sufficient pixels for face
)

8.5.2 Confidence Levels

8.6 Training Workflow Overview

The safe self-learning system captures operator feedback and converts it into model improvements through a carefully controlled pipeline.

8.6.1 Three Learning Modes

8.6.2 Training Pipeline Architecture

+=============================================================================+
|                    SAFE SELF-LEARNING PIPELINE                               |
+=============================================================================+
|                                                                              |
|  STEP 1: COLLECTION                                                          |
|  +-------------------+                                                       |
|  | Operator Review   |  confirm, correct_name, merge, reject                |
|  | Actions           |  + automatic high-confidence acceptances              |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  STEP 2: CONFLICT DETECTION (Synchronous, blocks immediately)               |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Label Conflict    | -> | If conflict found | -> | Block from training |   |
|  | Detector          |    | (5 types)         |    | dataset, alert admin |   |
|  | - Same face, diff |    +-------------------+    +-------------------+    |
|  |   names           |                                                       |
|  | - Diff faces, same|                                                       |
|  |   name            |                                                       |
|  | - Merge circular  |                                                       |
|  |   reference       |                                                       |
|  | - Name to already-|                                                       |
|  |   deleted person  |                                                       |
|  | - Quality below   |                                                       |
|  |   threshold       |                                                       |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  STEP 3: DATASET CURATION                                                    |
|  +-------------------+                                                       |
|  | Training Dataset  |  - Collect approved examples                         |
|  | Builder           |  - Balance classes (min 5 per person)                |
|  |                   |  - Augmentation (flip, rotate, brightness)           |
|  |                   |  - Quality filter (blur, pose, illumination)         |
|  |                   |  - Train/val split (80/20)                            |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  STEP 4: MODEL TRAINING                                                      |
|  +-------------------+                                                       |
|  | Training Job      |  - ArcFace R100 backbone                              |
|  | (Airflow DAG)     |  - Fine-tuning on curated dataset                     |
|  |                   |  - Cosine annealing LR schedule                        |
|  |                   |  - Early stopping (patience=10)                       |
|  |                   |  - Mixed precision (AMP)                              |
|  |                   |  - Typical duration: 2-8 hours on V100                |
|  +-------------------+                                                       |
|       |                                                                      |
|       v                                                                      |
|  STEP 5: QUALITY GATES                                                       |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Gate 1: Hold-out  | -> | Gate 2: Compare   | -> | Gate 3: Identity  |    |
|  |    evaluation     |    |    vs current     |    |    accuracy       |    |
|  |    (precision,    |    |    production     |    |    (100% known)   |    |
|  |     recall, f1)   |    |    (no >2% regress)|   |                   |    |
|  +-------------------+    +-------------------+    +-------------------+    |
|       |                          |                          |                |
|       +------------+-------------+--------------------------+                |
|                    |                                                          |
|         +----------+----------+                                              |
|         |                     |                                              |
|         v                     v                                              |
|     ALL PASSED            ANY FAILED                                       |
|         |                     |                                              |
|         v                     v                                              |
|  +------------+       +------------------+                                 |
|  | Proceed to |       | REJECT           |                                 |
|  | Deployment |       | - Log failure    |                                 |
|  +------------+       | - Alert admin    |                                 |
|                       | - Keep in staging|                                 |
|                       +------------------+                                 |
|                                                                              |
|  STEP 6: DEPLOYMENT                                                          |
|  +-------------------+                                                       |
|  | A/B Testing       |  - Shadow mode: 0% traffic (validation)              |
|  | (gradual rollout) |  - Canary: 5% traffic for 24h                        |
|  |                   |  - Monitor: latency, error rate, FP rate              |
|  |                   |  - Full rollout: 100% traffic                         |
|  |                   |  - Rollback: < 60 seconds to previous version         |
|  +-------------------+                                                       |
|                                                                              |
+=============================================================================+

8.7 Model Versioning and Rollback

8.7.1 Semantic Versioning

8.7.2 Version States

8.7.3 Rollback Procedure

+-----------------------------------------------------------------------------+
|                    EMERGENCY ROLLBACK PROCEDURE                              |
+-----------------------------------------------------------------------------+
|                                                                              |
|  Trigger: Admin initiates rollback or automatic rollback on failure         |
|                                                                              |
|  Step 1: Validate target version exists and is in ROLLBACK_AVAILABLE state  |
|  Step 2: Load target model artifacts from S3/MinIO (pre-warm GPU)          |
|  Step 3: Atomic switch: update model reference in Triton config             |
|  Step 4: Triton SIGHUP reload (zero-downtime model swap)                   |
|  Step 5: Validate: send test inference requests, check latency              |
|  Step 6: If validation fails -> auto-revert to previous production          |
|  Step 7: If validation passes -> update database model version records      |
|  Step 8: Log rollback event in audit_logs                                   |
|                                                                              |
|  Maximum rollback time: < 60 seconds                                        |
|  Zero inference downtime during rollback                                    |
|                                                                              |
+-----------------------------------------------------------------------------+

8.8 Quality Gates

8.8.1 Gate Thresholds

8.8.2 Quality Gate Report Example

{
  "gate_run_id": "550e8400-e29b-41d4-a716-446655440000",
  "candidate_model_version": "1.2.0",
  "baseline_model_version": "1.1.0",
  "timestamp": "2024-01-25T10:30:00Z",
  "overall_result": "PASSED",
  "gates": [
    {
      "name": "holdout_performance",
      "status": "PASSED",
      "critical": true,
      "metrics": {
        "precision": 0.9842,
        "recall": 0.9678,
        "f1_score": 0.9759
      }
    },
    {
      "name": "no_regression",
      "status": "PASSED",
      "metrics": {
        "max_regression_pct": 0.8,
        "per_metric": {
          "precision": 0.003,
          "recall": -0.008,
          "f1_score": -0.002
        }
      }
    },
    {
      "name": "known_identity_accuracy",
      "status": "PASSED",
      "metrics": {
        "known_identities_tested": 142,
        "perfect_accuracy": 142,
        "accuracy_below_threshold": 0
      }
    },
    {
      "name": "latency_requirement",
      "status": "PASSED",
      "metrics": {
        "p50_latency_ms": 45,
        "p99_latency_ms": 128,
        "threshold_ms": 150
      }
    }
  ]
}

8.8.3 Embedding Update Strategies

After a model passes quality gates and is deployed, the face embedding database must be updated. Five strategies are available:

Decision Matrix:

+-----------------------------------------------------------------------------+
|                    EMBEDDING UPDATE STRATEGY SELECTION                       |
+-----------------------------------------------------------------------------+
|                                                                              |
|  Model changed?                                                              |
|       |                                                                      |
|       +-- YES -> FULL_REINDEX (required, embeddings are model-dependent)     |
|       |                                                                      |
|       NO -> What changed?                                                    |
|               |                                                              |
|               +-- Identity merge -> MERGE_AND_UPDATE                         |
|               |                                                              |
|               +-- Rollback -> ROLLBACK_REINDEX                               |
|               |                                                              |
|               +-- New examples?                                              |
|                       |                                                      |
|                       +-- < 10 per identity, < 10% total -> CENTROID_UPDATE |
|                       |                                                      |
|                       +-- Otherwise -> INCREMENTAL_ADD                       |
|                                                                              |
+-----------------------------------------------------------------------------+

Reference: For complete model export commands, INT8 calibration scripts, performance benchmarks, and the full Python module structure, see ai_vision.md — Sections 10-14. For the complete training pipeline code, Airflow DAG definitions, and quality gate implementations, see training_system.md — Sections 5-10.

Section 9: Suspicious Activity Night-Mode Design

9.1 Overview

The suspicious activity detection system provides comprehensive behavioral analysis during night hours (22:00-06:00 by default) through 10 specialized detection modules. Each module operates on the output of the AI inference pipeline (detected persons, tracked positions, and face identities) to identify anomalous behavior patterns.

The system features a composite scoring engine that combines signals from all modules with exponential time-decay, enabling unified threat assessment and intelligent escalation. Each camera can be independently configured with custom zones, thresholds, and schedules.

9.2 Ten Detection Modules Summary

9.3 Module Details

9.3.1 Module 1: Intrusion Detection

Detects when a person enters a user-defined restricted polygon zone.

Algorithm:

For each detected person:
    For each restricted zone polygon:
        Compute IoU(person_bbox, zone_polygon)
        If IoU > overlap_threshold AND confidence > confidence_threshold:
            If zone not in cooldown:
                Trigger INTRUSION alert
                Start cooldown timer

9.3.2 Module 2: Loitering Detection

Flags persons who remain in an area longer than a threshold.

Algorithm:

For each active track:
    If track centroid moved < tolerance in last N seconds:
        Increment dwell timer
        If dwell_timer > threshold:
            Trigger LOITERING alert
            Reset timer (or hold until movement detected)
    Else:
        Reset dwell timer

9.3.3 Module 3: Running Detection

Identifies abnormally fast movement using pose keypoints and optical flow.

Algorithm:

For each active track:
    Compute torso keypoint displacement between frames
    Convert pixel speed to km/h (if calibration available)
    Apply Farneback optical flow for refinement
    If speed > threshold for confirmation_frames:
        Trigger RUNNING alert

9.3.4 Module 4: Crowding Detection

Alerts when person group density exceeds threshold.

Algorithm:

Collect all person centroids in current frame
Run DBSCAN(eps=0.08, min_samples=2) on centroids
For each cluster:
    If cluster_size >= count_threshold OR cluster_area >= area_threshold:
        Trigger CROWDING alert

9.3.5 Module 5: Fall Detection

Detects persons falling or collapsing using pose keypoint analysis.

Algorithm:

For each detected person with pose keypoints:
    Compute torso angle from vertical (using shoulder-hip line)
    Compute bbox aspect ratio
    Check if person is on ground (feet keypoint confidence drops)
    Calculate fall_score = weighted_combination(angle, aspect_ratio, ground_contact)
    If fall_score > threshold AND duration > confirmation_ms:
        Trigger FALL alert (CRITICAL severity)

9.3.6 Module 6: Abandoned Object Detection

Identifies unattended objects using background subtraction and object detection.

Algorithm:

Run YOLOv8s to detect objects in watchlist_classes
Run MOG2 background subtraction to identify static foreground
For each detected object:
    Track owner proximity (nearest person)
    If owner distance > threshold AND object stationary > time_threshold:
        Trigger ABANDONED_OBJECT alert

9.3.7 Module 7: After-Hours Presence

Simple but effective: any person detected during night hours triggers an alert.

9.3.8 Module 8: Zone Breach

Detects crossing of virtual boundary lines (directional or bidirectional).

Algorithm:

For each active track:
    For each boundary line:
        Check if track centroid crosses line in forbidden direction
        Using line equation: ax + by + c = 0, check sign change
        If crossed AND distance_past_line > threshold:
            Trigger ZONE_BREACH alert

9.3.9 Module 9: Repeated Re-entry Patterns

Detects suspicious patterns of entering and exiting an area multiple times.

State Machine:

For each track:
    Track state: OUTSIDE -> ENTERING -> INSIDE -> EXITING -> OUTSIDE
    Each complete cycle (entry + exit) increments counter
    If cycle_count >= threshold within time_window:
        Trigger REENTRY_PATTERN alert

9.3.10 Module 10: Suspicious Dwell Time

Extended presence near sensitive areas (different from general loitering).

Predefined zone types with default thresholds:

9.4 Activity Scoring Engine

9.4.1 Composite Score Formula

All 10 modules feed into a unified scoring engine that produces a single suspicious activity score per camera:

S_total(t) = SUM_i( weight_i * signal_i(t) * decay(t - t_i) ) + bonus_cross_module

Where:
    weight_i: module-specific weight (see table below)
    signal_i(t): normalized signal value from module i [0, 1]
    decay(delta_t): exponential time-decay function
    bonus_cross_module: extra score when multiple modules fire simultaneously
    t_i: timestamp of most recent event from module i

9.4.2 Module Weights

Note: Weights sum to 1.40 — this is intentional to allow cross-module amplification when multiple modules fire simultaneously.

9.4.3 Time-Decay Function

def time_decay(delta_t_seconds, half_life=300):
    """Exponential decay with 5-minute half-life by default."""
    import math
    return math.exp(-0.693 * delta_t_seconds / half_life)

# Decay reference:
#   0 min -> 1.000 (full contribution)
#   1 min -> 0.871
#   5 min -> 0.500
#  10 min -> 0.250
#  20 min -> 0.063
#  30 min -> 0.016 (effectively zero)

9.4.4 Cross-Module Amplification Bonus

When multiple modules detect simultaneously for the same track or in close proximity:

def compute_cross_module_bonus(active_signals, proximity_weight=0.15):
    n_modules = len(active_signals)
    if n_modules <= 1:
        return 0.0

    # Base bonus: +15% per additional module
    base_bonus = proximity_weight * (n_modules - 1)

    # Track overlap: same person triggering multiple rules -> higher threat
    track_bonus = 0.10 * (n_same_track_signals - 1) if n_same_track_signals >= 2 else 0

    # Zone overlap: multiple signals in same zone -> higher threat
    zone_bonus = 0.08 * (n_same_zone_signals - 1) if n_same_zone_signals >= 2 else 0

    return min(base_bonus + track_bonus + zone_bonus, 0.50)  # Cap at +0.50

9.4.5 Escalation Thresholds

9.5 Night Mode Scheduler

9.5.1 Automatic Schedule

9.5.2 Gradual Transition

During the 15-minute transition window, sensitivity ramps linearly:

Transition Start (21:45)          Night Full (22:00)         Transition End (22:15)
      |                                  |                           |
      v                                  v                           v
Sensitivity: 0% ---- 25% ---- 50% ---- 75% ---- 100% ---- 100% ---- 100%
              |__________|__________|__________|__________|__________|
                  Ramp up to full night sensitivity over 15 minutes

This prevents sudden spikes in alerts when night mode activates.

9.5.3 Night Mode Behavior Changes

9.6 Per-Camera Configuration

Each camera has independent configuration for all detection modules:

# Example: Camera 1 - Main Entrance
cam_01:
  enabled: true
  location: "Main Entrance Lobby"
  night_mode:
    enabled: true
    custom_schedule: null        # Use system default (22:00-06:00)
    sensitivity_multiplier: 1.0   # Standard sensitivity

  intrusion_detection:
    enabled: true
    confidence_threshold: 0.65
    overlap_threshold: 0.30
    cooldown_seconds: 30
    restricted_zones:
      - zone_id: "server_room_door"
        polygon: [[0.65,0.20], [0.85,0.20], [0.85,0.60], [0.65,0.60]]
        severity: "HIGH"

  loitering_detection:
    enabled: true
    dwell_time_threshold_seconds: 300
    movement_tolerance_pixels: 50

  running_detection:
    enabled: true
    speed_threshold_pixels_per_second: 150
    confirmation_frames: 3

  fall_detection:
    enabled: true
    fall_score_threshold: 0.75
    temporal_confirmation_ms: 1000

  # ... (all 10 modules configured)

9.7 Alert Generation Logic

9.7.1 Alert Lifecycle

+------------+    +------------+    +------------+    +------------+
|  DETECTED  | -> | SUPPRESSED | -> |  EVIDENCE  | -> | DISPATCHED |
| (Rule fire)|    | (Dedup)    |    | (Capture)  |    | (Send)     |
+------------+    +------------+    +------------+    +------------+
                                                          |
                                                          v
                                                   +------------+
                                                   | ACKNOWLEDGE|
                                                   | or AUTO    |
                                                   +------------+

9.7.2 Suppression Rules

9.7.3 Suppression Configuration

9.7.4 Severity Assignment

Final alert severity considers both the triggering module and the composite score context:

def assign_alert_severity(detection_event, composite_score):
    base_severity = detection_event['severity']  # From module config
    severity_levels = {'LOW': 1, 'MEDIUM': 2, 'HIGH': 3, 'CRITICAL': 4}
    base_level = severity_levels.get(base_severity, 2)

    # Escalation: high composite score bumps severity up one level
    if composite_score >= 0.80 and base_level < 3:
        base_level = min(base_level + 1, 4)

    # Escalation: multiple concurrent detections for same track
    if detection_event.get('concurrent_detections_count', 0) >= 2:
        base_level = min(base_level + 1, 4)

    # Zone-specific escalation override
    if detection_event.get('zone_severity_override'):
        zone_level = severity_levels.get(detection_event['zone_severity_override'], base_level)
        base_level = max(base_level, zone_level)

    reverse_levels = {v: k for k, v in severity_levels.items()}
    return reverse_levels.get(base_level, 'MEDIUM')

9.8 Integration with Main AI Pipeline

The suspicious activity service consumes detection events from the main AI pipeline:

+-----------------------------------------------------------------------------+
|               SUSPICIOUS ACTIVITY INTEGRATION WITH MAIN PIPELINE             |
+-----------------------------------------------------------------------------+
|                                                                              |
|  Main AI Pipeline Output:                                                    |
|  { person_id, track_id, bbox, keypoints, face_embedding, timestamp,        |
|    camera_id, confidence, face_crop_path }                                  |
|       |                                                                      |
|       v                                                                      |
|  +-------------------+    +-------------------+    +-------------------+    |
|  | Kafka Topic       | -> | Suspicious Activity| -> | Scoring Engine    |    |
|  | ai.detections     |    | Service            |    | (per camera)      |    |
|  | (JSON events)     |    | - 10 modules       |    | - Composite score |    |
|  +-------------------+    | - Per-camera config|    | - Time decay      |    |
|                           | - Zone polygons    |    | - Cross-module    |    |
|                           +-------------------+    |   bonus           |    |
|                                                     +---------+---------+    |
|                                                               |              |
|                                                               v              |
|                           +-------------------+    +-------------------+    |
|                           | Alert Manager     | <- | Scoring Output    |    |
|                           | - Deduplicate     |    | - Score [0, 1.5]  |    |
|                           | - Rate limit      |    | - Threat level    |    |
|                           | - Severity assign |    | - Active signals  |    |
|                           +---------+---------+    +-------------------+    |
|                                     |                                        |
|                                     v                                        |
|                           +-------------------+                             |
|                           | Alerts Table (DB) |                             |
|                           | Notification Svc  |                             |
|                           +-------------------+                             |
|                                                                              |
+-----------------------------------------------------------------------------+

Key integration points:

• Suspicious Activity Service is a Kafka consumer on the ai.detections topic
• Processes events after face recognition (has access to person identity)
• Produces alert records to the alerts.critical topic for notification dispatch
• Updates the composite score in Redis (with TTL = 2 * half_life) for dashboard real-time display
• Stores all alert records in PostgreSQL for history and analytics

Reference: For complete detection algorithm pseudocode, zone configuration YAML schema, scoring engine implementation, and evidence capture logic, see suspicious_activity.md — Sections 2-6.

Section 10: Live Video Streaming Design

10.1 RTSP Stream Configuration for CP PLUS DVR

10.1.1 URL Format

The CP PLUS ORANGE DVR uses a Dahua-compatible RTSP URL scheme:

rtsp://admin:{password}@{dvr_ip}:554/cam/realmonitor?channel={N}&subtype={M}

Where:
    N = channel number (1-8)
    M = stream type (0 = main stream, 1 = sub stream)

Example URLs for all 8 channels:

10.1.2 Stream Properties

10.1.3 Stream Discovery

The edge gateway can auto-discover streams via ONVIF:

from onvif import ONVIFCamera

camera = ONVIFCamera('192.168.29.200', 80, 'admin', 'password')
media_service = camera.create_media_service()
profiles = media_service.GetProfiles()

for profile in profiles:
    stream_uri = media_service.GetStreamUri({
        'StreamSetup': {'Stream': 'RTP_unicast', 'Transport': 'RTSP'},
        'ProfileToken': profile.token
    })
    print(f"Channel: {profile.token}, URI: {stream_uri.Uri}")

10.2 Edge Gateway Stream Handling

10.2.1 FFmpeg Ingestion Pipeline

The edge gateway runs one FFmpeg process per camera stream:

# Main stream: HLS generation for live viewing
ffmpeg -hide_banner -loglevel warning \
    -rtsp_transport tcp -stimeout 5000000 \
    -fflags +genpts+discardcorrupt+igndts+ignidx \
    -reorder_queue_size 64 -buffer_size 655360 \
    -i "rtsp://admin:password@192.168.29.200:554/cam/realmonitor?channel=1&subtype=0" \
    -c:v copy -c:a copy \
    -f hls -hls_time 2 -hls_list_size 5 -hls_delete_threshold 2 \
    -hls_flags delete_segments+omit_endlist+program_date_time \
    -hls_segment_filename "/data/hls/ch1_%04d.ts" \
    "/data/hls/ch1.m3u8" \
    2>> /var/log/ffmpeg_ch1.log

10.2.2 Stream Health Monitoring

10.2.3 Auto-Reconnect Logic

class StreamReconnectManager:
    """Handles RTSP stream reconnection with exponential backoff."""

    INITIAL_BACKOFF = 1.0       # seconds
    MAX_BACKOFF = 60.0          # seconds
    BACKOFF_MULTIPLIER = 2.0
    JITTER = 0.1                # 10% random jitter

    def __init__(self):
        self.current_backoff = self.INITIAL_BACKOFF
        self.consecutive_failures = 0

    def on_disconnect(self):
        self.consecutive_failures += 1
        wait_time = min(
            self.current_backoff * (self.BACKOFF_MULTIPLIER ** self.consecutive_failures),
            self.MAX_BACKOFF
        )
        # Add jitter to prevent thundering herd
        wait_time *= (1 + random.uniform(-self.JITTER, self.JITTER))
        return wait_time

    def on_success(self):
        self.current_backoff = self.INITIAL_BACKOFF
        self.consecutive_failures = 0

    def should_circuit_break(self):
        return self.consecutive_failures >= 5  # Open circuit after 5 failures

10.3 HLS Generation for Dashboard

10.3.1 HLS Segment Configuration

10.3.2 Multi-Bitrate HLS (Optional)

For adaptive bitrate streaming, three variants are generated per channel:

# High quality (main stream, copy codec)
ffmpeg -i "rtsp://...channel=1&subtype=0" -c:v copy -f hls -hls_time 2 \
    -hls_playlist_type vod -hls_segment_filename "ch1_high_%04d.ts" "ch1_high.m3u8"

# Medium quality (transcoded)
ffmpeg -i "rtsp://...channel=1&subtype=0" -c:v libx264 -preset fast -crf 23 \
    -vf "scale=640:480" -f hls -hls_time 2 \
    -hls_segment_filename "ch1_mid_%04d.ts" "ch1_mid.m3u8"

# Low quality (sub stream)
ffmpeg -i "rtsp://...channel=1&subtype=1" -c:v copy -f hls -hls_time 2 \
    -hls_segment_filename "ch1_low_%04d.ts" "ch1_low.m3u8"

Master playlist:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=4000000,RESOLUTION=960x1080
ch1_high.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=640x480
ch1_mid.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=500000,RESOLUTION=352x288
ch1_low.m3u8

10.3.3 HLS Latency Budget

10.4 WebRTC for Low-Latency Single Camera

For single-camera fullscreen viewing where low latency is critical, WebRTC provides sub-second delivery.

10.4.1 WebRTC Architecture

+------------+    +-------------------+    +-------------------+    +--------+
| Browser    |    | Edge Gateway      |    | FFmpeg            |    | DVR    |
| (WebRTC    |<-->| (WHIP/WHEP        |<-->| (decode RTSP,     |<-->| RTSP   |
|  client)   |    |  bridge)          |    |  encode VP8/H.264)|    | Server |
+------------+    +-------------------+    +-------------------+    +--------+

10.4.2 WebRTC Configuration

10.4.3 WebRTC Latency Budget

10.5 Multi-Camera Grid Layout

10.5.1 Layout Configurations

Smart stream selection: The dashboard automatically switches streams based on layout:

• Fullscreen single camera -> Main stream (high quality)
• Grid layout -> Sub stream (bandwidth-efficient)
• Camera clicked for fullscreen -> Dynamically switch to main stream

10.5.2 Grid Rendering

+-----------------------------------------------------------------------------+
|                         DASHBOARD GRID LAYOUTS                               |
+-----------------------------------------------------------------------------+
|                                                                              |
|  1x1 Layout:                         2x2 Layout:                            |
|  +------------------------+          +----------+----------+                 |
|  |                        |          | CH1      | CH2      |                 |
|  |   Camera 1             |          | (sub)    | (sub)    |                 |
|  |   Main stream          |          |          |          |                 |
|  |   960x1080             |          +----------+----------+                 |
|  |   ~4 Mbps              |          | CH3      | CH4      |                 |
|  +------------------------+          | (sub)    | (sub)    |                 |
|                                      |          |          |                 |
|                                      +----------+----------+                 |
|                                                                              |
|  3x3 Layout (8 cameras):                                                     |
|  +----------+----------+----------+                                          |
|  | CH1      | CH2      | CH3      |                                          |
|  | (sub)    | (sub)    | (sub)    |                                          |
|  +----------+----------+----------+                                          |
|  | CH4      | CH5      | CH6      |                                          |
|  | (sub)    | (sub)    | (sub)    |                                          |
|  +----------+----------+----------+                                          |
|  | CH7      | CH8      | [Empty]  |                                          |
|  | (sub)    | (sub)    |          |                                          |
|  +----------+----------+----------+                                          |
|                                                                              |
|  Bandwidth: ~8 Mbps total for 3x3 layout (8 x ~1 Mbps sub streams)          |
|                                                                              |
+-----------------------------------------------------------------------------+

10.6 Bandwidth Optimization

10.6.1 Total Bandwidth Budget

10.6.2 Optimization Techniques

10.7 Fallback Handling

10.7.1 Stream Failure Fallback Chain

Step 1: RTSP connection fails
    +-> Retry with exponential backoff (3 attempts)
    +-> Try UDP transport if TCP fails
    +-> Circuit breaker opens after 5 consecutive failures
    |
Step 2: Stream stall detected (no frames for 10s)
    +-> Kill FFmpeg process
    +-> Restart with fresh connection
    |
Step 3: Camera marked OFFLINE
    +-> Dashboard shows "Camera Offline" placeholder
    +-> HLS playlist returns 404
    +-> Last known frame displayed with timestamp overlay
    +-> Alert sent to operations team
    |
Step 4: Camera recovers
    +-> Circuit breaker transitions to HALF_OPEN
    +-> Test stream pulled for 10 seconds
    +-> On success: circuit CLOSED, stream resumes
    +-> Dashboard auto-refreshes

10.7.2 Offline Placeholder

When a camera is offline, the HLS endpoint returns a static playlist:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ERROR: "Camera OFFLINE - Channel 1"
#EXTINF:2.000,
offline_placeholder.ts

The dashboard detects the #EXT-X-ERROR tag and displays a camera offline indicator with the last known timestamp.

10.7.3 Edge Buffer Management

The 2TB NVMe edge storage is partitioned for circular buffer operation:

Buffer exhaustion handling:

1. At 80% capacity: Alert admin, begin aggressive cleanup of old non-evidence data
2. At 90% capacity: Stop non-critical buffering (AI frames), preserve HLS + evidence only
3. At 95% capacity: Emergency mode — evidence-only recording, all other buffers purged
4. Never delete evidence clips linked to unresolved alerts

10.7.4 DVR Full Disk Mitigation

Since the DVR disk is full (0 bytes free), the system does not rely on DVR-side recording:

Reference: For complete FFmpeg commands including multi-output tee muxer, frame extraction for AI, WebRTC bridge code, and the ring buffer implementation, see video_ingestion.md — Sections 4-7.

End of Part A (Sections 1-10)

This unified technical blueprint synthesizes outputs from 11 specialist agents across 6 domain-specific design documents. For detailed implementation code, DDL, algorithms, and configuration, refer to the individual specialist documents listed in the cross-reference guide at the top of this document.

Sentinel AI Surveillance Platform — Unified Technical Blueprint (Part B)

Document Version: 1.0 Date: 2025-01-16 Classification: Confidential — Internal Use Only Author: Technical Architecture Team

Part B Table of Contents

• Section 11: Alerting Design (Notification System)
• Section 12: Security Design
• Section 13: UX / Website Structure
• Section 14: Deployment Plan
• Section 15: Testing Plan
• Section 16: Self-Test Framework
• Section 17: Sample Self-Test Report
• Section 18: Risks and Mitigations
• Section 19: Final Implementation Roadmap
• Section 20: Final Production-Readiness Summary

Section 11: Alerting Design

11.1 Architecture Overview

The notification system employs an event-driven architecture built on Redis Pub/Sub for real-time message distribution. All detection events, system alerts, and manual triggers flow through a unified pipeline that supports dual-channel delivery via Telegram Bot API and WhatsApp Business API (Meta Official). The system is designed to ensure that critical security alerts are never lost while maintaining high performance and reliability through sophisticated rate limiting, retry logic, and dead letter queue handling.

┌──────────────────────────────────────────────────────────────────────────────┐
│                         ALERTING ARCHITECTURE                                 │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                         EVENT SOURCES                            │       │
│   │                                                                  │       │
│   │  Detection Pipeline ──▶ New person detected                     │       │
│   │  Face Recognition ────▶ Known/Unknown/Watchlist match           │       │
│   │  System Monitors ─────▶ Camera offline, Storage full, VPN down   │       │
│   │  Manual Triggers ─────▶ Operator-initiated alerts                │       │
│   │  AI Anomaly Engine ───▶ Suspicious activity detected             │       │
│   └──────────────────────────┬──────────────────────────────────────┘       │
│                              │                                               │
│                              ▼                                               │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                    REDIS PUB/SUB                                 │       │
│   │                                                                  │       │
│   │  Channel: alerts.critical  ─── High priority, immediate process  │       │
│   │  Channel: alerts.high      ─── Standard priority                 │       │
│   │  Channel: alerts.medium    ─── Batched processing                │       │
│   │  Channel: system.health    ─── System health events              │       │
│   └──────────────┬───────────────────────────────────────────────────┘       │
│                  │                                                            │
│   ┌──────────────┴───────────────────────────────────────────────────┐       │
│   │                    NOTIFICATION ROUTER                           │       │
│   │                     (Python/FastAPI)                             │       │
│   │                                                                  │       │
│   │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │       │
│   │  │ Event Parser │──▶ Rules Engine │──▶ Channel Selector    │  │       │
│   │  └──────────────┘  └──────────────┘  └──────────────────────┘  │       │
│   └──────────────────────────┬───────────────────────────────────────┘       │
│                              │                                               │
│          ┌───────────────────┼───────────────────┐                           │
│          ▼                   ▼                   ▼                           │
│   ┌─────────────┐   ┌───────────────┐   ┌──────────────────┐               │
│   │   TEMPLATE  │   │    RATE       │   │   ESCALATION     │               │
│   │   RENDERER  │   │   LIMITER     │   │    ENGINE        │               │
│   │             │   │               │   │                  │               │
│   │  HTML/TXT   │   │ Token Bucket  │   │ 3-level timeout  │               │
│   │  per channel│   │ 4-tier limits │   │ Auto-escalation  │               │
│   └──────┬──────┘   └───────┬───────┘   └────────┬─────────┘               │
│          │                  │                     │                         │
│          └──────────────────┼─────────────────────┘                         │
│                             ▼                                               │
│   ┌─────────────────────────────────────────────────────────────────┐      │
│   │                    CHANNEL ADAPTERS                              │      │
│   │                                                                  │      │
│   │  ┌──────────────────────────┐  ┌──────────────────────────┐     │      │
│   │  │    TELEGRAM BOT API      │  │  WHATSAPP BUSINESS API   │     │      │
│   │  │                          │  │                          │     │      │
│   │  │  - HTML formatting       │  │  - Template messages     │     │      │
│   │  │  - Inline keyboards      │  │  - Session messages      │     │      │
│   │  │  - Media groups          │  │  - Interactive messages  │     │      │
│   │  │  - Edit/Delete messages  │  │  - Media attachments     │     │      │
│   │  │  - Webhook receipts      │  │  - Message status API    │     │      │
│   │  └──────────┬───────────────┘  └──────────┬───────────────┘     │      │
│   └─────────────┼─────────────────────────────┼─────────────────────┘      │
│                 │                             │                              │
│                 ▼                             ▼                              │
│          ┌──────────────┐            ┌──────────────┐                       │
│          │  Telegram    │            │  WhatsApp    │                       │
│          │  Servers     │            │  Cloud API   │                       │
│          └──────────────┘            └──────────────┘                       │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                    SUPPORTING SERVICES                           │       │
│   │                                                                  │       │
│   │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │       │
│   │  │ RETRY MGR    │  │    DLQ       │  │  DELIVERY TRACKER    │  │       │
│   │  │ Exponential  │  │ Redis-backed │  │  Webhook callbacks   │  │       │
│   │  │ 5 max        │  │ Admin review │  │  Status dashboard    │  │       │
│   │  └──────────────┘  └──────────────┘  └──────────────────────┘  │       │
│   └─────────────────────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────────────────────┘

Key Design Principles:

11.2 Telegram Integration

11.2.1 Bot API Configuration

Telegram integration uses the official Telegram Bot API for message delivery. The bot is configured with encrypted tokens stored in HashiCorp Vault, with HTML message formatting for rich alert presentation.

11.2.2 Bot Features and Capabilities

Inline Keyboards: Every alert message includes contextual action buttons that allow operators to respond directly from Telegram without opening the web dashboard.

Media Groups: When an alert contains multiple evidence images (up to 10), they are sent as a Telegram media group (album). This presents all related images in a single scrollable gallery rather than individual messages, reducing chat clutter.

Webhook Receipts: Telegram delivers message status updates via webhooks:

Chat Commands:

11.2.3 Security Considerations

Telegram bot tokens are among the most sensitive credentials in the system. The following security measures are implemented:

11.3 WhatsApp Business API Integration

11.3.1 Meta Cloud API Configuration

WhatsApp integration uses Meta's official Cloud API (Business Platform), which provides a reliable, enterprise-grade messaging channel. This requires a verified Meta Business account and pre-approved message templates for proactive messaging.

11.3.2 Message Types

Template Messages: Pre-approved message templates are required for any proactive (business-initiated) message. Templates must be created and submitted for approval in Meta Business Manager. Each template contains named parameters that are dynamically populated at send time.

Session Messages: Within a 24-hour window after a user sends a message to the business, free-form session messages can be sent without template restrictions. This is used for:

• Acknowledgment confirmations
• Escalation follow-ups
• Interactive conversations initiated by the recipient
• Quick reply responses

11.3.3 Webhook Event Handling

11.4 Alert Routing Rules Engine

11.4.1 Condition Types

The routing engine evaluates 9 distinct condition types to determine which recipients receive which alerts through which channels. Multiple conditions can be combined with AND/OR logic for precise targeting.

11.4.2 Rule Structure

Each routing rule consists of conditions, actions, and metadata:

rule:
  id: "rule-001"
  name: "Blacklist Immediate Alert"
  enabled: true
  priority: 100  # Higher number = evaluated first
  
  conditions:
    operator: "AND"
    conditions:
      - field: "watchlist"
        operator: "equals"
        value: "blacklist"
      - field: "severity"
        operator: "in"
        value: ["critical", "high"]
  
  actions:
    - channel: "telegram"
      recipients: ["security_team", "management"]
      template: "blacklist_alert"
      media: ["image", "video"]
      bypass_quiet_hours: true
      priority: "high"
    
    - channel: "whatsapp"
      recipients: ["security_manager"]
      template: "blacklist_alert"
      media: ["image"]
      bypass_quiet_hours: true
  
  metadata:
    created_by: "admin"
    created_at: "2025-01-01T00:00:00Z"
    last_modified: "2025-01-10T12:00:00Z"
    tags: ["critical", "blacklist"]

11.4.3 Default Routing Rules

The system ships with a comprehensive set of default routing rules that cover common surveillance scenarios:

11.5 Recipient Groups and Quiet Hours

11.5.1 Recipient Group Management

Recipient groups are the primary mechanism for organizing alert destinations. Each group contains one or more contacts with specified channels.

Group Configuration Interface:

Groups are managed through the web dashboard at /settings/notifications/groups. Each group can be configured with:

11.5.2 Quiet Hours Configuration

Quiet hours allow suppressing non-critical alerts during configured time windows. Critical alerts always bypass quiet hours — this is a non-configurable safety measure.

quiet_hours:
  enabled: false                    # DISABLED BY DEFAULT for security
  preset: "none"                    # none / nights / weekends / custom
  
  custom_schedule:
    - label: "Weekday Nights"
      days: ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
      start_time: "22:00"
      end_time: "06:00"
      timezone: "Asia/Kolkata"
      
    - label: "Weekend All Day"
      days: ["Saturday", "Sunday"]
      start_time: "00:00"
      end_time: "23:59"
      timezone: "Asia/Kolkata"
  
  allowed_during_quiet:             # Which severities bypass quiet hours
    - "critical"                    # Always delivered (non-configurable)
    
  emergency_bypass:
    enabled: true
    triggers:
      - severity: "critical"
      - tag: "emergency"
      - rule_override: "bypass_quiet_hours"
    notification_method: "all_channels"
    
  suppression_behavior: "queue"     # queue / discard / digest
  # "queue": Hold until quiet hours end
  # "discard": Drop non-critical alerts entirely
  # "digest": Send summary when quiet hours end

Security Note: Quiet hours are disabled by default because the surveillance use case requires continuous awareness. Any decision to enable quiet hours must be documented with security team sign-off.

11.5.3 Per-Recipient Quiet Hours

Individual recipients can configure personal quiet hours that override group settings:

11.6 Message Templates

11.6.1 Telegram HTML Templates

All Telegram templates use a safe HTML subset for rich formatting with inline action keyboards.

Template: Person Detected (Known)

🔍 <b>Person Detected</b>

<b>{name}</b> ({role})
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<a href="{dashboard_url}">View in Dashboard</a>

Template: Unknown Person Detected

❓ <b>Unknown Person Detected</b>

📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<i>This person is not in the database.</i>

<a href="{naming_url}">Name This Person</a>

Template: Watchlist Match

⚠️ <b>WATCHLIST ALERT</b>

<b>{name}</b>
📋 Watchlist: {watchlist_type}
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<i>This person is on a watchlist and requires attention.</i>

Template: Blacklist Alert

🚨 <b>BLACKLIST ALERT</b> 🚨

⚠️ <b>{name}</b> has been detected!
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<b>This person is BLACKLISTED. Immediate attention required.</b>

<a href="{dispatch_url}">🚨 Dispatch Security</a>

Template: Escalation Notice

⬆️ <b>Alert Escalated — Level {escalation_level}</b>

Alert #{alert_id} has been escalated.

Original: {alert_summary}
⏱️ Unacknowledged for {elapsed_minutes} minutes
Threshold: {threshold_minutes} minutes

<i>Please review immediately.</i>

Template: System Alert

⚙️ <b>System Alert</b>

{message}

🕐 {timestamp}
Severity: {severity}

<a href="{health_dashboard_url}">View System Health</a>

Template: Daily Digest

📊 <b>Daily Activity Digest — {date}</b>

👥 Persons Detected: {total_detections}
🔔 Alerts Generated: {total_alerts}
📹 Cameras Online: {cameras_online}/{cameras_total}

Top Cameras:
{camera_list}

<a href="{full_report_url}">View Full Report</a>

11.6.2 WhatsApp Template Format

WhatsApp templates use a different format — they are pre-registered with Meta and use numbered parameter substitution:

Template: person_detected_known

🔍 Person Detected

{{1}} ({{2}})
📍 Camera: {{3}}
🕐 {{4}} at {{5}}
🎯 Confidence: {{6}}%
Alert ID: {{7}}

Parameters: {{1}}=name, {{2}}=role, {{3}}=camera_name, {{4}}=date, {{5}}=time, {{6}}=confidence, {{7}}=alert_id

11.6.3 Template Variable Reference

11.7 Retry Logic and Rate Limiting

11.7.1 Retry Configuration

Failed notifications are retried using an exponential backoff strategy to avoid overwhelming downstream services.

Retry Schedule:

Retryable Errors:

Non-Retryable Errors (Immediate DLQ):

• Invalid bot token (401)
• Bot blocked by user (403)
• Chat not found
• Malformed template (400)
• Message too long (after split)
• Unsupported media format

11.7.2 Circuit Breaker

Each channel adapter implements a circuit breaker to prevent cascading failures:

Circuit States:

11.7.3 Rate Limiting Tiers

The notification system implements multi-tier rate limiting to prevent abuse and ensure fair resource distribution:

Token Bucket Algorithm: Each tier maintains a token bucket. A token is consumed per message. Tokens replenish at the configured rate. If no tokens are available, the message is queued or rejected based on priority.

11.7.4 Alert Deduplication

Alerts are deduplicated to prevent notification spam when the same event triggers repeatedly:

When a duplicate is detected, the original message is updated with a counter (e.g., "+3 more detections"), avoiding a flood of similar messages.

11.8 Escalation Rules

11.8.1 Escalation Thresholds

When an alert goes unacknowledged, it automatically escalates through up to 3 levels, each with increasing urgency and broader recipient distribution.

11.8.2 Escalation Actions per Level

Escalation Cancellation: Acknowledgment cancels ALL pending escalation timers for an alert. Acknowledgment can occur via:

• Telegram inline "Acknowledge" button click
• WhatsApp quick reply "Ack"
• Web dashboard "Acknowledge" button
• REST API POST /api/v1/alerts/{id}/acknowledge
• Chat command /acknowledge {alert_id}

11.8.3 Escalation Notification Template

⬆️ <b>ESCALATION — Level {level}</b>

Original Alert: {alert_summary}
Alert ID: {alert_id}
First Detected: {first_detected_time}
Current Time: {current_time}
Unacknowledged: {elapsed_minutes} minutes
Escalation Threshold: {threshold_minutes} minutes

This alert has been escalated because it has not been acknowledged.
Please review immediately.

<a href="{acknowledge_url}">✅ Acknowledge Now</a>
<a href="{view_alert_url}">👁 View Details</a>

11.9 Media Attachment Handling

11.9.1 Media Processing Pipeline

When an alert includes media (snapshot images or video clips), a multi-stage processing pipeline ensures the media meets channel-specific requirements:

Original Media (from detection)
       │
       ▼
┌──────────────────┐
│  1. Store Original│  ──▶ MinIO/S3 (full resolution archival)
│     in Storage    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  2. Process for  │
│     Telegram     │
└────────┬─────────┘
         │
         ├──▶ Image: Resize 1280x720, JPEG quality 85, max 10 MB
         ├──▶ Video: H.264, 1280x720, max 50 MB, max 60 seconds
         └──▶ Media Group: Each image < 10 MB, max 10 items
         │
         ▼
┌──────────────────┐
│  3. Process for  │
│     WhatsApp     │
└────────┬─────────┘
         │
         ├──▶ Image: Resize 1600x900, JPEG quality 80, max 16 MB
         └──▶ Video: H.264, 1280x720, max 16 MB, max 60 seconds

11.9.2 Image Processing Details

Iterative Quality Reduction:

def compress_image_to_limit(image, size_limit_mb, channel):
    quality = 85 if channel == 'telegram' else 80
    min_quality = 40
    
    while quality >= min_quality:
        buffer = io.BytesIO()
        image.save(buffer, format='JPEG', quality=quality, optimize=True)
        size_mb = buffer.tell() / (1024 * 1024)
        
        if size_mb <= size_limit_mb:
            return buffer.getvalue()
        
        quality -= 5
    
    # If still over limit, reduce dimensions by 25% and retry
    new_size = (int(image.width * 0.75), int(image.height * 0.75))
    image = image.resize(new_size, Image.LANCZOS)
    return compress_image_to_limit(image, size_limit_mb, channel)

11.9.3 Video Processing Details

Videos are processed with FFmpeg using two-pass encoding to achieve the target bitrate calculated from the size limit:

# Calculate target bitrate: (size_limit_bytes * 8) / duration_seconds
# Example: 16 MB limit, 10 second clip = (16*1024*1024*8) / 10 = ~13.4 Mbps

ffmpeg -i input.mp4 \
    -c:v libx264 \
    -b:v 10M \                          # Target video bitrate
    -maxrate 12M \                      # Maximum bitrate
    -bufsize 20M \                      # Buffer size
    -vf "scale=1280:720:force_original_aspect_ratio=decrease" \
    -c:a aac -b:a 128k \               # Audio encoding
    -movflags +faststart \              # Web-optimized
    -preset fast \                      # Encoding speed/quality tradeoff
    -y output.mp4

11.10 Delivery Tracking

11.10.1 Delivery Status Lifecycle

Every notification progresses through a well-defined status lifecycle, tracked in the database for audit and troubleshooting:

Status Transitions:

pending → processing → sent → delivered → read → engaged
   │          │          │         │
   ▼          ▼          ▼         ▼
retrying  cancelled   failed   suppressed
   │
   ▼
dead_letter

11.10.2 Dead Letter Queue (DLQ)

Failed notifications that exhaust all retry attempts are moved to a Redis-backed Dead Letter Queue. Admin users can review and manage DLQ entries through the web dashboard.

11.11 API Endpoints Summary

11.11.1 REST Endpoints (13 endpoints)

11.11.2 Alert Management Endpoints

11.11.3 WebSocket Endpoints (2 endpoints)

11.11.4 Webhook Endpoints (2 endpoints)

Webhook Security:

Section 12: Security Design

12.1 Security Architecture Overview

The Sentinel AI Surveillance Platform implements defense-in-depth security across seven distinct layers. Every component — from network perimeter to data storage — has been designed with security as a primary consideration, reflecting the sensitive nature of surveillance data, biometric information, and the critical safety function the system performs.

┌──────────────────────────────────────────────────────────────────────────────┐
│                      DEFENSE IN DEPTH ARCHITECTURE                            │
│                                                                              │
│   LAYER 1: PERIMETER                                                         │
│   ─────────────────                                                          │
│   AWS WAF v2 │ Geo-restriction │ DDoS protection │ Rate limiting             │
│                                                                              │
│   LAYER 2: TRANSPORT                                                         │
│   ─────────────────                                                          │
│   TLS 1.3 │ mTLS internal │ WireGuard ChaCha20-Poly1305 │ Certificate mgmt  │
│                                                                              │
│   LAYER 3: AUTHENTICATION & AUTHORIZATION                                    │
│   ─────────────────────────────────────────                                  │
│   Argon2id │ JWT ES256 │ TOTP MFA │ RBAC 4 roles │ API keys                 │
│                                                                              │
│   LAYER 4: APPLICATION SECURITY                                              │
│   ────────────────────────────                                               │
│   Input validation │ Parameterized queries │ CSP │ CSRF │ CORS │ File upload │
│                                                                              │
│   LAYER 5: DATA SECURITY                                                     │
│   ────────────────────                                                       │
│   AES-256-GCM at rest │ Field-level encryption │ Signed URLs │ Key rotation  │
│                                                                              │
│   LAYER 6: NETWORK SEGMENTATION                                              │
│   ───────────────────────────                                                │
│   VPC private subnets │ Security groups │ Network Policies │ Firewall rules  │
│                                                                              │
│   LAYER 7: AUDIT & MONITORING                                                │
│   ─────────────────────────                                                  │
│   Hash-chain audit log │ Real-time alerts │ CloudTrail │ Flow Logs           │
└──────────────────────────────────────────────────────────────────────────────┘

12.2 SSL/TLS Configuration

12.2.1 Protocol and Cipher Suite Requirements

All external-facing services enforce strong TLS configuration with modern cipher suites:

12.2.2 mTLS for Internal Service Communication

All inter-service communication uses mutual TLS (mTLS) with client certificate verification. This means both the client and server must present valid certificates signed by the internal Certificate Authority.

Benefits of mTLS:

• Even if network boundaries are breached, unauthorized services cannot access internal APIs
• Every service-to-service call is authenticated and encrypted
• Certificates provide strong service identity (not just IP-based)
• No shared secrets between services (except Vault tokens)

12.2.3 TLS Configuration Code Example

# FastAPI TLS configuration
from fastapi import FastAPI
from uvicorn.config import Config

app = FastAPI()

# TLS settings for uvicorn
ssl_config = {
    "ssl_keyfile": "/certs/server.key",
    "ssl_certfile": "/certs/server.crt",
    "ssl_ca_certs": "/certs/ca.crt",          # For mTLS
    "ssl_cert_reqs": ssl.CERT_REQUIRED,        # Require client cert
    "ssl_min_version": ssl.TLSVersion.TLSv1_2,
    "ssl_ciphers": "ECDHE-ECDSA-AES256-GCM-SHA384:"
                    "ECDHE-RSA-AES256-GCM-SHA384:"
                    "ECDHE-ECDSA-CHACHA20-POLY1305:"
                    "ECDHE-RSA-CHACHA20-POLY1305",
}

12.3 Authentication

12.3.1 Password Policy

12.3.2 JWT Token Configuration

JWT Token Structure:

{
  "header": {
    "alg": "ES256",
    "typ": "JWT",
    "kid": "key-2025-01"
  },
  "payload": {
    "sub": "user-uuid-here",
    "iss": "sentinel-ai",
    "aud": "sentinel-api",
    "exp": 1705500000,
    "iat": 1705499100,
    "jti": "unique-token-id",
    "role": "operator",
    "permissions": ["alerts:view", "alerts:acknowledge", "cameras:view"],
    "mfa_verified": true,
    "session_id": "sess-uuid-here"
  }
}

12.3.3 Multi-Factor Authentication (MFA)

MFA Enforcement Matrix:

12.4 Role-Based Access Control (RBAC)

12.4.1 Role Definitions

12.4.2 Permission Matrix (30+ Permissions)

12.4.3 Resource-Level Permissions

Beyond global permissions, the system supports resource-level access control:

12.5 VPN and Network Security

12.5.1 WireGuard VPN Configuration

WireGuard provides the encrypted tunnel between cloud infrastructure and the edge site:

12.5.2 Network Segmentation Architecture

┌──────────────────────────────────────────────────────────────────────────────┐
│                         NETWORK ARCHITECTURE                                  │
│                                                                              │
│   INTERNET                                                                   │
│      │                                                                       │
│      ▼                                                                       │
│   ┌──────────────┐                                                           │
│   │  AWS WAF     │                                                           │
│   │  + ALB       │                                                           │
│   └──────┬───────┘                                                           │
│          │                                                                   │
│   ═══════╪════════════════ AWS CLOUD VPC: 10.100.0.0/16 ═══════════════════  │
│          │                                                                   │
│          │    ┌──────────────────────────────────────────────────────┐       │
│          │    │  PUBLIC SUBNET: 10.100.1.0/24                      │       │
│          │    │  - ALB (Application Load Balancer)                  │       │
│          │    │  - NAT Gateway                                       │       │
│          │    │  - WireGuard VPN Gateway (10.200.0.1)               │       │
│          │    │  - Bastion Host (emergency SSH, admin IPs only)     │       │
│          │    └──────────────────────────────────────────────────────┘       │
│          │                                                                   │
│          └────▶┌──────────────────────────────────────────────────────┐      │
│               │  PRIVATE SUBNET: 10.100.2.0/24 (App Tier)            │      │
│               │  - EKS Worker Nodes (API, AI, Web pods)              │      │
│               │  - Stream Ingestion Service                          │      │
│               │  - Alert Engine                                      │      │
│               │  - Notification Service                              │      │
│               └──────────────────────────────────────────────────────┘      │
│               ▲                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  DATA SUBNET: 10.100.3.0/24 (No Internet)          │ │
│               │    │  - RDS PostgreSQL (Multi-AZ)                        │ │
│               │    │  - ElastiCache Redis Cluster                        │ │
│               │    │  - Amazon MSK Kafka                                 │ │
│               │    │  - NO INTERNET ACCESS (VPC endpoints only)          │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  MONITORING SUBNET: 10.100.4.0/24                  │ │
│               │    │  - Prometheus, Grafana, Alertmanager                │ │
│               │    │  - Loki (log aggregation)                           │ │
│               │    │  - Jaeger (distributed tracing)                     │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                                                              │
│   ════════════╪══════════════════════════════════════════════════════════    │
│               │                                                              │
│               │         WireGuard VPN Tunnel (UDP 51820)                   │
│               │                                                              │
│   ════════════╪══════════════════════════════════════════════════════════    │
│               │                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  EDGE GATEWAY: 192.168.29.5/24 (Intel NUC)           │ │
│               │    │  OS: Ubuntu Server 22.04 LTS (minimal)               │ │
│               │    │  - Docker Compose stack                              │ │
│               │    │  - WireGuard Client (10.200.0.2)                     │ │
│               │    │  - Local MinIO (hot storage)                         │ │
│               │    │  - Redis (local cache)                               │ │
│               │    │  - Video Capture Service                             │ │
│               │    │  - AI Inference (edge models)                        │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                              │                               │
│               │    ┌─────────────────────────┴──────────────────────┐       │
│               │    │  CAMERA LAN: 192.168.29.0/24                    │       │
│               │    │  - CP PLUS DVR: 192.168.29.200 (8 channels)     │       │
│               │    │  - RTSP streams on port 554                     │       │
│               │    │  - NO INTERNET ACCESS                           │       │
│               │    │  - NO ROUTE TO CLOUD (only via edge gateway)    │       │
│               │    └────────────────────────────────────────────────┘       │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

12.5.3 Firewall Rules

Edge Gateway Firewall (iptables):

Cloud Firewall (AWS Security Groups):

12.6 Secret Management

12.6.1 Vault Integration

All secrets are stored in HashiCorp Vault with automatic rotation policies:

12.6.2 Dynamic Database Credentials

Instead of static database passwords, the system uses Vault's dynamic credential engine:

Application → Vault (request db credentials)
                  │
                  ▼
           Vault creates temporary DB user
           (TTL: 1 hour, auto-revoke)
                  │
                  ▼
           Application receives credentials
           Uses them for DB connections
                  │
                  ▼
           After TTL expires → Vault revokes DB user
           Application requests new credentials

Benefits:

• No long-lived database passwords in application configuration
• Each application instance gets unique credentials
• Automatic credential rotation without application restart
• Full audit trail of credential issuance and revocation
• Instant credential revocation on compromise

12.6.3 Field-Level Encryption

PII and biometric data in the database uses AES-256-GCM field-level encryption:

Encryption Architecture:

Application receives plaintext data
       │
       ▼
[Encrypt field-by-field using Vault KMS]
       │
       ▼
Store ciphertext in PostgreSQL
       │
       ▼
[Decrypt only in application layer when needed]
       │
       ▼
Decrypted data never logged, never cached

12.7 Audit Logging

12.7.1 Tamper-Resistant Hash-Chain

The audit log implements a cryptographically linked chain to ensure integrity:

Chain Verification: Any modification to historical entries invalidates all subsequent hashes and signatures, making tampering detectable.

12.7.2 Log Retention Policy

12.7.3 Real-Time Security Alerting

Automated detection rules trigger alerts on suspicious patterns:

12.8 Media Access Security

12.8.1 Signed URL Architecture

Media files are never served directly from object storage. All access is mediated through signed URLs:

Signed URL Flow:

1. User requests to view media
2. System checks: authentication + authorization + consent
3. If allowed: generate signed URL with HMAC-SHA256 signature
4. URL format: https://cdn.example.com/media/{id}?token={jwt}&sig={hmac}
5. Redirect user to signed URL
6. CDN/Object storage validates signature and expiry
7. Media served if valid; 403 if expired or invalid
8. Access logged with full context

12.8.2 Media Access Controls

12.9 API Security

12.9.1 Defense Layers

12.9.2 Security Headers

12.10 Session Security

12.11 Data Privacy (GDPR Compliance)

12.11.1 GDPR Compliance Matrix

12.11.2 Consent Management

Consent is managed through a comprehensive lifecycle:

Consent Metadata:

Withdrawal Processing:

1. User submits withdrawal request (any channel)
2. System flags person record for deletion
3. Delete face embeddings (biometric data) within 72 hours
4. Delete all personal images from storage
5. Anonymize detection events (keep event, replace name with [REDACTED], remove person link)
6. Delete related event clips
7. Log all deletion actions in audit trail
8. Confirm completion to user within 30 days

12.11.3 Privacy Mode Controls

Four privacy modes are available per camera:

12.12 Edge Gateway Security

12.12.1 Hardening Checklist

12.12.2 LUKS Disk Encryption with TPM2

The edge gateway uses LUKS full-disk encryption with TPM2 auto-unseal for headless operation:

# During setup — encrypt the data partition
cryptsetup luksFormat /dev/nvme0n1p2 \
    --type luks2 \
    --cipher aes-xts-plain64 \
    --key-size 512 \
    --pbkdf argon2id \
    --tpm2-device=auto

# Bind the LUKS key to TPM2 PCR measurements
cryptsetup luksAddKey /dev/nvme0n1p2 \
    --key-slot 1 \
    --tpm2-device=auto \
    --tpm2-pcrs=0,2,7

# During boot — TPM2 auto-unseals if PCRs match
cryptsetup open --tpm2-device=auto /dev/nvme0n1p2 data

PCR Measurements Bound:

12.13 Cloud Infrastructure Security