Sentinel AI Surveillance Platform — Unified Technical Blueprint (Part B)

Document Version: 1.0 Date: 2025-01-16 Classification: Confidential — Internal Use Only Author: Technical Architecture Team

Part B Table of Contents

• Section 11: Alerting Design (Notification System)
• Section 12: Security Design
• Section 13: UX / Website Structure
• Section 14: Deployment Plan
• Section 15: Testing Plan
• Section 16: Self-Test Framework
• Section 17: Sample Self-Test Report
• Section 18: Risks and Mitigations
• Section 19: Final Implementation Roadmap
• Section 20: Final Production-Readiness Summary

Section 11: Alerting Design

11.1 Architecture Overview

The notification system employs an event-driven architecture built on Redis Pub/Sub for real-time message distribution. All detection events, system alerts, and manual triggers flow through a unified pipeline that supports dual-channel delivery via Telegram Bot API and WhatsApp Business API (Meta Official). The system is designed to ensure that critical security alerts are never lost while maintaining high performance and reliability through sophisticated rate limiting, retry logic, and dead letter queue handling.

┌──────────────────────────────────────────────────────────────────────────────┐
│                         ALERTING ARCHITECTURE                                 │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                         EVENT SOURCES                            │       │
│   │                                                                  │       │
│   │  Detection Pipeline ──▶ New person detected                     │       │
│   │  Face Recognition ────▶ Known/Unknown/Watchlist match           │       │
│   │  System Monitors ─────▶ Camera offline, Storage full, VPN down   │       │
│   │  Manual Triggers ─────▶ Operator-initiated alerts                │       │
│   │  AI Anomaly Engine ───▶ Suspicious activity detected             │       │
│   └──────────────────────────┬──────────────────────────────────────┘       │
│                              │                                               │
│                              ▼                                               │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                    REDIS PUB/SUB                                 │       │
│   │                                                                  │       │
│   │  Channel: alerts.critical  ─── High priority, immediate process  │       │
│   │  Channel: alerts.high      ─── Standard priority                 │       │
│   │  Channel: alerts.medium    ─── Batched processing                │       │
│   │  Channel: system.health    ─── System health events              │       │
│   └──────────────┬───────────────────────────────────────────────────┘       │
│                  │                                                            │
│   ┌──────────────┴───────────────────────────────────────────────────┐       │
│   │                    NOTIFICATION ROUTER                           │       │
│   │                     (Python/FastAPI)                             │       │
│   │                                                                  │       │
│   │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │       │
│   │  │ Event Parser │──▶ Rules Engine │──▶ Channel Selector    │  │       │
│   │  └──────────────┘  └──────────────┘  └──────────────────────┘  │       │
│   └──────────────────────────┬───────────────────────────────────────┘       │
│                              │                                               │
│          ┌───────────────────┼───────────────────┐                           │
│          ▼                   ▼                   ▼                           │
│   ┌─────────────┐   ┌───────────────┐   ┌──────────────────┐               │
│   │   TEMPLATE  │   │    RATE       │   │   ESCALATION     │               │
│   │   RENDERER  │   │   LIMITER     │   │    ENGINE        │               │
│   │             │   │               │   │                  │               │
│   │  HTML/TXT   │   │ Token Bucket  │   │ 3-level timeout  │               │
│   │  per channel│   │ 4-tier limits │   │ Auto-escalation  │               │
│   └──────┬──────┘   └───────┬───────┘   └────────┬─────────┘               │
│          │                  │                     │                         │
│          └──────────────────┼─────────────────────┘                         │
│                             ▼                                               │
│   ┌─────────────────────────────────────────────────────────────────┐      │
│   │                    CHANNEL ADAPTERS                              │      │
│   │                                                                  │      │
│   │  ┌──────────────────────────┐  ┌──────────────────────────┐     │      │
│   │  │    TELEGRAM BOT API      │  │  WHATSAPP BUSINESS API   │     │      │
│   │  │                          │  │                          │     │      │
│   │  │  - HTML formatting       │  │  - Template messages     │     │      │
│   │  │  - Inline keyboards      │  │  - Session messages      │     │      │
│   │  │  - Media groups          │  │  - Interactive messages  │     │      │
│   │  │  - Edit/Delete messages  │  │  - Media attachments     │     │      │
│   │  │  - Webhook receipts      │  │  - Message status API    │     │      │
│   │  └──────────┬───────────────┘  └──────────┬───────────────┘     │      │
│   └─────────────┼─────────────────────────────┼─────────────────────┘      │
│                 │                             │                              │
│                 ▼                             ▼                              │
│          ┌──────────────┐            ┌──────────────┐                       │
│          │  Telegram    │            │  WhatsApp    │                       │
│          │  Servers     │            │  Cloud API   │                       │
│          └──────────────┘            └──────────────┘                       │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────┐       │
│   │                    SUPPORTING SERVICES                           │       │
│   │                                                                  │       │
│   │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │       │
│   │  │ RETRY MGR    │  │    DLQ       │  │  DELIVERY TRACKER    │  │       │
│   │  │ Exponential  │  │ Redis-backed │  │  Webhook callbacks   │  │       │
│   │  │ 5 max        │  │ Admin review │  │  Status dashboard    │  │       │
│   │  └──────────────┘  └──────────────┘  └──────────────────────┘  │       │
│   └─────────────────────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────────────────────┘

Key Design Principles:

11.2 Telegram Integration

11.2.1 Bot API Configuration

Telegram integration uses the official Telegram Bot API for message delivery. The bot is configured with encrypted tokens stored in HashiCorp Vault, with HTML message formatting for rich alert presentation.

11.2.2 Bot Features and Capabilities

Inline Keyboards: Every alert message includes contextual action buttons that allow operators to respond directly from Telegram without opening the web dashboard.

Media Groups: When an alert contains multiple evidence images (up to 10), they are sent as a Telegram media group (album). This presents all related images in a single scrollable gallery rather than individual messages, reducing chat clutter.

Webhook Receipts: Telegram delivers message status updates via webhooks:

Chat Commands:

11.2.3 Security Considerations

Telegram bot tokens are among the most sensitive credentials in the system. The following security measures are implemented:

11.3 WhatsApp Business API Integration

11.3.1 Meta Cloud API Configuration

WhatsApp integration uses Meta's official Cloud API (Business Platform), which provides a reliable, enterprise-grade messaging channel. This requires a verified Meta Business account and pre-approved message templates for proactive messaging.

11.3.2 Message Types

Template Messages: Pre-approved message templates are required for any proactive (business-initiated) message. Templates must be created and submitted for approval in Meta Business Manager. Each template contains named parameters that are dynamically populated at send time.

Session Messages: Within a 24-hour window after a user sends a message to the business, free-form session messages can be sent without template restrictions. This is used for:

• Acknowledgment confirmations
• Escalation follow-ups
• Interactive conversations initiated by the recipient
• Quick reply responses

11.3.3 Webhook Event Handling

11.4 Alert Routing Rules Engine

11.4.1 Condition Types

The routing engine evaluates 9 distinct condition types to determine which recipients receive which alerts through which channels. Multiple conditions can be combined with AND/OR logic for precise targeting.

11.4.2 Rule Structure

Each routing rule consists of conditions, actions, and metadata:

rule:
  id: "rule-001"
  name: "Blacklist Immediate Alert"
  enabled: true
  priority: 100  # Higher number = evaluated first
  
  conditions:
    operator: "AND"
    conditions:
      - field: "watchlist"
        operator: "equals"
        value: "blacklist"
      - field: "severity"
        operator: "in"
        value: ["critical", "high"]
  
  actions:
    - channel: "telegram"
      recipients: ["security_team", "management"]
      template: "blacklist_alert"
      media: ["image", "video"]
      bypass_quiet_hours: true
      priority: "high"
    
    - channel: "whatsapp"
      recipients: ["security_manager"]
      template: "blacklist_alert"
      media: ["image"]
      bypass_quiet_hours: true
  
  metadata:
    created_by: "admin"
    created_at: "2025-01-01T00:00:00Z"
    last_modified: "2025-01-10T12:00:00Z"
    tags: ["critical", "blacklist"]

11.4.3 Default Routing Rules

The system ships with a comprehensive set of default routing rules that cover common surveillance scenarios:

11.5 Recipient Groups and Quiet Hours

11.5.1 Recipient Group Management

Recipient groups are the primary mechanism for organizing alert destinations. Each group contains one or more contacts with specified channels.

Group Configuration Interface:

Groups are managed through the web dashboard at /settings/notifications/groups. Each group can be configured with:

11.5.2 Quiet Hours Configuration

Quiet hours allow suppressing non-critical alerts during configured time windows. Critical alerts always bypass quiet hours — this is a non-configurable safety measure.

quiet_hours:
  enabled: false                    # DISABLED BY DEFAULT for security
  preset: "none"                    # none / nights / weekends / custom
  
  custom_schedule:
    - label: "Weekday Nights"
      days: ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
      start_time: "22:00"
      end_time: "06:00"
      timezone: "Asia/Kolkata"
      
    - label: "Weekend All Day"
      days: ["Saturday", "Sunday"]
      start_time: "00:00"
      end_time: "23:59"
      timezone: "Asia/Kolkata"
  
  allowed_during_quiet:             # Which severities bypass quiet hours
    - "critical"                    # Always delivered (non-configurable)
    
  emergency_bypass:
    enabled: true
    triggers:
      - severity: "critical"
      - tag: "emergency"
      - rule_override: "bypass_quiet_hours"
    notification_method: "all_channels"
    
  suppression_behavior: "queue"     # queue / discard / digest
  # "queue": Hold until quiet hours end
  # "discard": Drop non-critical alerts entirely
  # "digest": Send summary when quiet hours end

Security Note: Quiet hours are disabled by default because the surveillance use case requires continuous awareness. Any decision to enable quiet hours must be documented with security team sign-off.

11.5.3 Per-Recipient Quiet Hours

Individual recipients can configure personal quiet hours that override group settings:

11.6 Message Templates

11.6.1 Telegram HTML Templates

All Telegram templates use a safe HTML subset for rich formatting with inline action keyboards.

Template: Person Detected (Known)

🔍 <b>Person Detected</b>

<b>{name}</b> ({role})
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<a href="{dashboard_url}">View in Dashboard</a>

Template: Unknown Person Detected

❓ <b>Unknown Person Detected</b>

📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<i>This person is not in the database.</i>

<a href="{naming_url}">Name This Person</a>

Template: Watchlist Match

⚠️ <b>WATCHLIST ALERT</b>

<b>{name}</b>
📋 Watchlist: {watchlist_type}
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<i>This person is on a watchlist and requires attention.</i>

Template: Blacklist Alert

🚨 <b>BLACKLIST ALERT</b> 🚨

⚠️ <b>{name}</b> has been detected!
📍 Camera: {camera_name}
🕐 {date} at {time}
🎯 Confidence: {confidence}%

<b>This person is BLACKLISTED. Immediate attention required.</b>

<a href="{dispatch_url}">🚨 Dispatch Security</a>

Template: Escalation Notice

⬆️ <b>Alert Escalated — Level {escalation_level}</b>

Alert #{alert_id} has been escalated.

Original: {alert_summary}
⏱️ Unacknowledged for {elapsed_minutes} minutes
Threshold: {threshold_minutes} minutes

<i>Please review immediately.</i>

Template: System Alert

⚙️ <b>System Alert</b>

{message}

🕐 {timestamp}
Severity: {severity}

<a href="{health_dashboard_url}">View System Health</a>

Template: Daily Digest

📊 <b>Daily Activity Digest — {date}</b>

👥 Persons Detected: {total_detections}
🔔 Alerts Generated: {total_alerts}
📹 Cameras Online: {cameras_online}/{cameras_total}

Top Cameras:
{camera_list}

<a href="{full_report_url}">View Full Report</a>

11.6.2 WhatsApp Template Format

WhatsApp templates use a different format — they are pre-registered with Meta and use numbered parameter substitution:

Template: person_detected_known

🔍 Person Detected

{{1}} ({{2}})
📍 Camera: {{3}}
🕐 {{4}} at {{5}}
🎯 Confidence: {{6}}%
Alert ID: {{7}}

Parameters: {{1}}=name, {{2}}=role, {{3}}=camera_name, {{4}}=date, {{5}}=time, {{6}}=confidence, {{7}}=alert_id

11.6.3 Template Variable Reference

11.7 Retry Logic and Rate Limiting

11.7.1 Retry Configuration

Failed notifications are retried using an exponential backoff strategy to avoid overwhelming downstream services.

Retry Schedule:

Retryable Errors:

Non-Retryable Errors (Immediate DLQ):

• Invalid bot token (401)
• Bot blocked by user (403)
• Chat not found
• Malformed template (400)
• Message too long (after split)
• Unsupported media format

11.7.2 Circuit Breaker

Each channel adapter implements a circuit breaker to prevent cascading failures:

Circuit States:

11.7.3 Rate Limiting Tiers

The notification system implements multi-tier rate limiting to prevent abuse and ensure fair resource distribution:

Token Bucket Algorithm: Each tier maintains a token bucket. A token is consumed per message. Tokens replenish at the configured rate. If no tokens are available, the message is queued or rejected based on priority.

11.7.4 Alert Deduplication

Alerts are deduplicated to prevent notification spam when the same event triggers repeatedly:

When a duplicate is detected, the original message is updated with a counter (e.g., "+3 more detections"), avoiding a flood of similar messages.

11.8 Escalation Rules

11.8.1 Escalation Thresholds

When an alert goes unacknowledged, it automatically escalates through up to 3 levels, each with increasing urgency and broader recipient distribution.

11.8.2 Escalation Actions per Level

Escalation Cancellation: Acknowledgment cancels ALL pending escalation timers for an alert. Acknowledgment can occur via:

• Telegram inline "Acknowledge" button click
• WhatsApp quick reply "Ack"
• Web dashboard "Acknowledge" button
• REST API POST /api/v1/alerts/{id}/acknowledge
• Chat command /acknowledge {alert_id}

11.8.3 Escalation Notification Template

⬆️ <b>ESCALATION — Level {level}</b>

Original Alert: {alert_summary}
Alert ID: {alert_id}
First Detected: {first_detected_time}
Current Time: {current_time}
Unacknowledged: {elapsed_minutes} minutes
Escalation Threshold: {threshold_minutes} minutes

This alert has been escalated because it has not been acknowledged.
Please review immediately.

<a href="{acknowledge_url}">✅ Acknowledge Now</a>
<a href="{view_alert_url}">👁 View Details</a>

11.9 Media Attachment Handling

11.9.1 Media Processing Pipeline

When an alert includes media (snapshot images or video clips), a multi-stage processing pipeline ensures the media meets channel-specific requirements:

Original Media (from detection)
       │
       ▼
┌──────────────────┐
│  1. Store Original│  ──▶ MinIO/S3 (full resolution archival)
│     in Storage    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  2. Process for  │
│     Telegram     │
└────────┬─────────┘
         │
         ├──▶ Image: Resize 1280x720, JPEG quality 85, max 10 MB
         ├──▶ Video: H.264, 1280x720, max 50 MB, max 60 seconds
         └──▶ Media Group: Each image < 10 MB, max 10 items
         │
         ▼
┌──────────────────┐
│  3. Process for  │
│     WhatsApp     │
└────────┬─────────┘
         │
         ├──▶ Image: Resize 1600x900, JPEG quality 80, max 16 MB
         └──▶ Video: H.264, 1280x720, max 16 MB, max 60 seconds

11.9.2 Image Processing Details

Iterative Quality Reduction:

def compress_image_to_limit(image, size_limit_mb, channel):
    quality = 85 if channel == 'telegram' else 80
    min_quality = 40
    
    while quality >= min_quality:
        buffer = io.BytesIO()
        image.save(buffer, format='JPEG', quality=quality, optimize=True)
        size_mb = buffer.tell() / (1024 * 1024)
        
        if size_mb <= size_limit_mb:
            return buffer.getvalue()
        
        quality -= 5
    
    # If still over limit, reduce dimensions by 25% and retry
    new_size = (int(image.width * 0.75), int(image.height * 0.75))
    image = image.resize(new_size, Image.LANCZOS)
    return compress_image_to_limit(image, size_limit_mb, channel)

11.9.3 Video Processing Details

Videos are processed with FFmpeg using two-pass encoding to achieve the target bitrate calculated from the size limit:

# Calculate target bitrate: (size_limit_bytes * 8) / duration_seconds
# Example: 16 MB limit, 10 second clip = (16*1024*1024*8) / 10 = ~13.4 Mbps

ffmpeg -i input.mp4 \
    -c:v libx264 \
    -b:v 10M \                          # Target video bitrate
    -maxrate 12M \                      # Maximum bitrate
    -bufsize 20M \                      # Buffer size
    -vf "scale=1280:720:force_original_aspect_ratio=decrease" \
    -c:a aac -b:a 128k \               # Audio encoding
    -movflags +faststart \              # Web-optimized
    -preset fast \                      # Encoding speed/quality tradeoff
    -y output.mp4

11.10 Delivery Tracking

11.10.1 Delivery Status Lifecycle

Every notification progresses through a well-defined status lifecycle, tracked in the database for audit and troubleshooting:

Status Transitions:

pending → processing → sent → delivered → read → engaged
   │          │          │         │
   ▼          ▼          ▼         ▼
retrying  cancelled   failed   suppressed
   │
   ▼
dead_letter

11.10.2 Dead Letter Queue (DLQ)

Failed notifications that exhaust all retry attempts are moved to a Redis-backed Dead Letter Queue. Admin users can review and manage DLQ entries through the web dashboard.

11.11 API Endpoints Summary

11.11.1 REST Endpoints (13 endpoints)

11.11.2 Alert Management Endpoints

11.11.3 WebSocket Endpoints (2 endpoints)

11.11.4 Webhook Endpoints (2 endpoints)

Webhook Security:

Section 12: Security Design

12.1 Security Architecture Overview

The Sentinel AI Surveillance Platform implements defense-in-depth security across seven distinct layers. Every component — from network perimeter to data storage — has been designed with security as a primary consideration, reflecting the sensitive nature of surveillance data, biometric information, and the critical safety function the system performs.

┌──────────────────────────────────────────────────────────────────────────────┐
│                      DEFENSE IN DEPTH ARCHITECTURE                            │
│                                                                              │
│   LAYER 1: PERIMETER                                                         │
│   ─────────────────                                                          │
│   AWS WAF v2 │ Geo-restriction │ DDoS protection │ Rate limiting             │
│                                                                              │
│   LAYER 2: TRANSPORT                                                         │
│   ─────────────────                                                          │
│   TLS 1.3 │ mTLS internal │ WireGuard ChaCha20-Poly1305 │ Certificate mgmt  │
│                                                                              │
│   LAYER 3: AUTHENTICATION & AUTHORIZATION                                    │
│   ─────────────────────────────────────────                                  │
│   Argon2id │ JWT ES256 │ TOTP MFA │ RBAC 4 roles │ API keys                 │
│                                                                              │
│   LAYER 4: APPLICATION SECURITY                                              │
│   ────────────────────────────                                               │
│   Input validation │ Parameterized queries │ CSP │ CSRF │ CORS │ File upload │
│                                                                              │
│   LAYER 5: DATA SECURITY                                                     │
│   ────────────────────                                                       │
│   AES-256-GCM at rest │ Field-level encryption │ Signed URLs │ Key rotation  │
│                                                                              │
│   LAYER 6: NETWORK SEGMENTATION                                              │
│   ───────────────────────────                                                │
│   VPC private subnets │ Security groups │ Network Policies │ Firewall rules  │
│                                                                              │
│   LAYER 7: AUDIT & MONITORING                                                │
│   ─────────────────────────                                                  │
│   Hash-chain audit log │ Real-time alerts │ CloudTrail │ Flow Logs           │
└──────────────────────────────────────────────────────────────────────────────┘

12.2 SSL/TLS Configuration

12.2.1 Protocol and Cipher Suite Requirements

All external-facing services enforce strong TLS configuration with modern cipher suites:

12.2.2 mTLS for Internal Service Communication

All inter-service communication uses mutual TLS (mTLS) with client certificate verification. This means both the client and server must present valid certificates signed by the internal Certificate Authority.

Benefits of mTLS:

• Even if network boundaries are breached, unauthorized services cannot access internal APIs
• Every service-to-service call is authenticated and encrypted
• Certificates provide strong service identity (not just IP-based)
• No shared secrets between services (except Vault tokens)

12.2.3 TLS Configuration Code Example

# FastAPI TLS configuration
from fastapi import FastAPI
from uvicorn.config import Config

app = FastAPI()

# TLS settings for uvicorn
ssl_config = {
    "ssl_keyfile": "/certs/server.key",
    "ssl_certfile": "/certs/server.crt",
    "ssl_ca_certs": "/certs/ca.crt",          # For mTLS
    "ssl_cert_reqs": ssl.CERT_REQUIRED,        # Require client cert
    "ssl_min_version": ssl.TLSVersion.TLSv1_2,
    "ssl_ciphers": "ECDHE-ECDSA-AES256-GCM-SHA384:"
                    "ECDHE-RSA-AES256-GCM-SHA384:"
                    "ECDHE-ECDSA-CHACHA20-POLY1305:"
                    "ECDHE-RSA-CHACHA20-POLY1305",
}

12.3 Authentication

12.3.1 Password Policy

12.3.2 JWT Token Configuration

JWT Token Structure:

{
  "header": {
    "alg": "ES256",
    "typ": "JWT",
    "kid": "key-2025-01"
  },
  "payload": {
    "sub": "user-uuid-here",
    "iss": "sentinel-ai",
    "aud": "sentinel-api",
    "exp": 1705500000,
    "iat": 1705499100,
    "jti": "unique-token-id",
    "role": "operator",
    "permissions": ["alerts:view", "alerts:acknowledge", "cameras:view"],
    "mfa_verified": true,
    "session_id": "sess-uuid-here"
  }
}

12.3.3 Multi-Factor Authentication (MFA)

MFA Enforcement Matrix:

12.4 Role-Based Access Control (RBAC)

12.4.1 Role Definitions

12.4.2 Permission Matrix (30+ Permissions)

12.4.3 Resource-Level Permissions

Beyond global permissions, the system supports resource-level access control:

12.5 VPN and Network Security

12.5.1 WireGuard VPN Configuration

WireGuard provides the encrypted tunnel between cloud infrastructure and the edge site:

12.5.2 Network Segmentation Architecture

┌──────────────────────────────────────────────────────────────────────────────┐
│                         NETWORK ARCHITECTURE                                  │
│                                                                              │
│   INTERNET                                                                   │
│      │                                                                       │
│      ▼                                                                       │
│   ┌──────────────┐                                                           │
│   │  AWS WAF     │                                                           │
│   │  + ALB       │                                                           │
│   └──────┬───────┘                                                           │
│          │                                                                   │
│   ═══════╪════════════════ AWS CLOUD VPC: 10.100.0.0/16 ═══════════════════  │
│          │                                                                   │
│          │    ┌──────────────────────────────────────────────────────┐       │
│          │    │  PUBLIC SUBNET: 10.100.1.0/24                      │       │
│          │    │  - ALB (Application Load Balancer)                  │       │
│          │    │  - NAT Gateway                                       │       │
│          │    │  - WireGuard VPN Gateway (10.200.0.1)               │       │
│          │    │  - Bastion Host (emergency SSH, admin IPs only)     │       │
│          │    └──────────────────────────────────────────────────────┘       │
│          │                                                                   │
│          └────▶┌──────────────────────────────────────────────────────┐      │
│               │  PRIVATE SUBNET: 10.100.2.0/24 (App Tier)            │      │
│               │  - EKS Worker Nodes (API, AI, Web pods)              │      │
│               │  - Stream Ingestion Service                          │      │
│               │  - Alert Engine                                      │      │
│               │  - Notification Service                              │      │
│               └──────────────────────────────────────────────────────┘      │
│               ▲                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  DATA SUBNET: 10.100.3.0/24 (No Internet)          │ │
│               │    │  - RDS PostgreSQL (Multi-AZ)                        │ │
│               │    │  - ElastiCache Redis Cluster                        │ │
│               │    │  - Amazon MSK Kafka                                 │ │
│               │    │  - NO INTERNET ACCESS (VPC endpoints only)          │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  MONITORING SUBNET: 10.100.4.0/24                  │ │
│               │    │  - Prometheus, Grafana, Alertmanager                │ │
│               │    │  - Loki (log aggregation)                           │ │
│               │    │  - Jaeger (distributed tracing)                     │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                                                              │
│   ════════════╪══════════════════════════════════════════════════════════    │
│               │                                                              │
│               │         WireGuard VPN Tunnel (UDP 51820)                   │
│               │                                                              │
│   ════════════╪══════════════════════════════════════════════════════════    │
│               │                                                              │
│               │    ┌──────────────────────────────────────────────────────┐ │
│               │    │  EDGE GATEWAY: 192.168.29.5/24 (Intel NUC)           │ │
│               │    │  OS: Ubuntu Server 22.04 LTS (minimal)               │ │
│               │    │  - Docker Compose stack                              │ │
│               │    │  - WireGuard Client (10.200.0.2)                     │ │
│               │    │  - Local MinIO (hot storage)                         │ │
│               │    │  - Redis (local cache)                               │ │
│               │    │  - Video Capture Service                             │ │
│               │    │  - AI Inference (edge models)                        │ │
│               │    └──────────────────────────────────────────────────────┘ │
│               │                              │                               │
│               │    ┌─────────────────────────┴──────────────────────┐       │
│               │    │  CAMERA LAN: 192.168.29.0/24                    │       │
│               │    │  - CP PLUS DVR: 192.168.29.200 (8 channels)     │       │
│               │    │  - RTSP streams on port 554                     │       │
│               │    │  - NO INTERNET ACCESS                           │       │
│               │    │  - NO ROUTE TO CLOUD (only via edge gateway)    │       │
│               │    └────────────────────────────────────────────────┘       │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

12.5.3 Firewall Rules

Edge Gateway Firewall (iptables):

Cloud Firewall (AWS Security Groups):

12.6 Secret Management

12.6.1 Vault Integration

All secrets are stored in HashiCorp Vault with automatic rotation policies:

12.6.2 Dynamic Database Credentials

Instead of static database passwords, the system uses Vault's dynamic credential engine:

Application → Vault (request db credentials)
                  │
                  ▼
           Vault creates temporary DB user
           (TTL: 1 hour, auto-revoke)
                  │
                  ▼
           Application receives credentials
           Uses them for DB connections
                  │
                  ▼
           After TTL expires → Vault revokes DB user
           Application requests new credentials

Benefits:

• No long-lived database passwords in application configuration
• Each application instance gets unique credentials
• Automatic credential rotation without application restart
• Full audit trail of credential issuance and revocation
• Instant credential revocation on compromise

12.6.3 Field-Level Encryption

PII and biometric data in the database uses AES-256-GCM field-level encryption:

Encryption Architecture:

Application receives plaintext data
       │
       ▼
[Encrypt field-by-field using Vault KMS]
       │
       ▼
Store ciphertext in PostgreSQL
       │
       ▼
[Decrypt only in application layer when needed]
       │
       ▼
Decrypted data never logged, never cached

12.7 Audit Logging

12.7.1 Tamper-Resistant Hash-Chain

The audit log implements a cryptographically linked chain to ensure integrity:

Chain Verification: Any modification to historical entries invalidates all subsequent hashes and signatures, making tampering detectable.

12.7.2 Log Retention Policy

12.7.3 Real-Time Security Alerting

Automated detection rules trigger alerts on suspicious patterns:

12.8 Media Access Security

12.8.1 Signed URL Architecture

Media files are never served directly from object storage. All access is mediated through signed URLs:

Signed URL Flow:

1. User requests to view media
2. System checks: authentication + authorization + consent
3. If allowed: generate signed URL with HMAC-SHA256 signature
4. URL format: https://cdn.example.com/media/{id}?token={jwt}&sig={hmac}
5. Redirect user to signed URL
6. CDN/Object storage validates signature and expiry
7. Media served if valid; 403 if expired or invalid
8. Access logged with full context

12.8.2 Media Access Controls

12.9 API Security

12.9.1 Defense Layers

12.9.2 Security Headers

12.10 Session Security

12.11 Data Privacy (GDPR Compliance)

12.11.1 GDPR Compliance Matrix

12.11.2 Consent Management

Consent is managed through a comprehensive lifecycle:

Consent Metadata:

Withdrawal Processing:

1. User submits withdrawal request (any channel)
2. System flags person record for deletion
3. Delete face embeddings (biometric data) within 72 hours
4. Delete all personal images from storage
5. Anonymize detection events (keep event, replace name with [REDACTED], remove person link)
6. Delete related event clips
7. Log all deletion actions in audit trail
8. Confirm completion to user within 30 days

12.11.3 Privacy Mode Controls

Four privacy modes are available per camera:

12.12 Edge Gateway Security

12.12.1 Hardening Checklist

12.12.2 LUKS Disk Encryption with TPM2

The edge gateway uses LUKS full-disk encryption with TPM2 auto-unseal for headless operation:

# During setup — encrypt the data partition
cryptsetup luksFormat /dev/nvme0n1p2 \
    --type luks2 \
    --cipher aes-xts-plain64 \
    --key-size 512 \
    --pbkdf argon2id \
    --tpm2-device=auto

# Bind the LUKS key to TPM2 PCR measurements
cryptsetup luksAddKey /dev/nvme0n1p2 \
    --key-slot 1 \
    --tpm2-device=auto \
    --tpm2-pcrs=0,2,7

# During boot — TPM2 auto-unseals if PCRs match
cryptsetup open --tpm2-device=auto /dev/nvme0n1p2 data

PCR Measurements Bound:

12.13 Cloud Infrastructure Security

12.14 Secrets Rotation Policy

12.15 Incident Response

12.15.1 Security Event Detection and Response

12.15.2 Breach Notification Procedure

12.15.3 Breach Severity Classification

12.16 Security Checklist Summary

The complete security checklist contains 100+ items across 15 categories. The following table summarizes the key items per category:

Section 13: UX / Website Structure

13.1 Design System

13.1.1 Design Philosophy

The UX design follows a "dark cockpit" philosophy optimized for 24/7 surveillance operations. The interface minimizes eye strain during long monitoring shifts while ensuring critical information is immediately visible. All design decisions prioritize operator efficiency and rapid threat identification.

13.1.2 Color Palette

13.1.3 Typography

13.1.4 Spacing and Layout

13.2 Global Navigation Structure

13.2.1 Layout Architecture

┌──────────────────────────────────────────────────────────────────────────────┐
│ [Logo]  Sentinel AI Surveillance              [Clock] [Alerts] [👤 User] │  ▲ 56px
├────────┬───────────────────────────────────────────────────────────────────┤
│        │                                                                    │
│  [📊]  │                    MAIN CONTENT AREA                              │
│  Dash  │                                                                    │
│  board │    ┌──────────────┐  ┌──────────────┐  ┌──────────────┐         │
│        │    │   Card 1     │  │   Card 2     │  │   Card 3     │         │
│  [📹]  │    │              │  │              │  │              │         │
│  Live  │    └──────────────┘  └──────────────┘  └──────────────┘         │
│        │                                                                    │
│  [🔔]  │    ┌──────────────────────────────────────────────────┐         │
│ Alerts │    │              Wide Card / Table                   │         │
│        │    └──────────────────────────────────────────────────┘         │
│  [🔍]  │                                                                    │
│ Detec  │                                                                    │
│ tions  │                                                                    │
│        │                                                                    │
│ [remaining navigation items...]                                            │
│        │                                                                    │
├────────┤                                                                    │
│◁ / ▷  │                                                                    │
└────────┴───────────────────────────────────────────────────────────────────┘
  ◄── 260px (expanded) / 72px (collapsed) ──►

13.2.2 Navigation Menu Items

Settings Submenu:

13.2.3 Top Bar

User Menu Dropdown:

13.3 Page Descriptions

13.3.1 Page 1: Login (/login)

The login page is the entry point to the system. It is designed for quick, secure access with minimal friction.

13.3.2 Page 2: Dashboard (/dashboard)

The Dashboard is the primary landing page providing at-a-glance situational awareness.

┌──────────────────────────────────────────────────────────────────────────────┐
│  Dashboard                                          [Refresh] [Date Range] │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ ┌──────────────┐ │
│  │  📹 8/8       │ │  🔔 12        │ │  👥 47        │ │  ✓ Healthy  │ │
│  │  Cameras      │ │  Alerts Today │ │  Persons      │ │  System     │ │
│  │  Online       │ │  3 Critical   │ │  Detected     │ │  All Good   │ │
│  └────────────────┘ └────────────────┘ └────────────────┘ └──────────────┘ │
│                                                                              │
│  ┌────────────────────────────────────────┐ ┌──────────────────────────┐   │
│  │  Alert Distribution (Last 24 Hours)   │ │  Recent Alerts           │   │
│  │                                        │ │                          │   │
│  │  8 ┤          ██                       │ │  🔴 CAM-01  Unknown     │   │
│  │  6 ┤    ██    ██  ██                   │ │     14:32 — Entrance    │   │
│  │  4 ┤    ██ ██ ██  ██ ██                │ │  🟡 CAM-03  Watchlist   │   │
│  │  2 ┤ ██ ██ ██ ██  ██ ██ ██             │ │     13:15 — Parking     │   │
│  │  0 ┼────┬────┬────┬────┬────┬────┬──  │ │  🟠 CAM-05  System      │   │
│  │     00  04  08  12  16  20           │ │     12:08 — Storage 90% │   │
│  │                                        │ │                          │   │
│  └────────────────────────────────────────┘ └──────────────────────────┘   │
│                                                                              │
│  ┌──────────────────────────────────────────────────────────────────┐     │
│  │  Camera Status Grid (2x4)                                        │     │
│  │                                                                    │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐           │     │
│  │  │ CAM-01  │ │ CAM-02  │ │ CAM-03  │ │ CAM-04  │           │     │
│  │  │ [LIVE]  │ │ [LIVE]  │ │ [LIVE]  │ │ [LIVE]  │           │     │
│  │  │ ● Online│ │ ● Online│ │ ● Online│ │ ● Online│           │     │
│  │  └──────────┘ └──────────┘ └──────────┘ └──────────┘           │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐           │     │
│  │  │ CAM-05  │ │ CAM-06  │ │ CAM-07  │ │ CAM-08  │           │     │
│  │  │ [LIVE]  │ │ [LIVE]  │ │ [LIVE]  │ │ [LIVE]  │           │     │
│  │  │ ● Online│ │ ● Online│ │ ● Online│ │ ● Online│           │     │
│  │  └──────────┘ └──────────┘ └──────────┘ └──────────┘           │     │
│  └──────────────────────────────────────────────────────────────────┘     │
│                                                                              │
│  ┌──────────────────────────────────────────────────────────────────┐     │
│  │  Activity Feed                                                    │     │
│  │  14:32 — Unknown person detected at CAM-01 (Entrance)            │     │
│  │  14:15 — Watchlist match: John Smith at CAM-03 (Parking)         │     │
│  │  13:58 — Operator Alice acknowledged alert #ALT-2847             │     │
│  │  13:42 — Camera CAM-05 stream reconnected                        │     │
│  │  13:30 — Daily training completed: 3 new face clusters          │     │
│  └──────────────────────────────────────────────────────────────────┘     │
└──────────────────────────────────────────────────────────────────────────────┘

Dashboard Components:

13.3.3 Page 3: Live Camera View (/live)

The live view is the primary monitoring interface, showing real-time streams from all 8 cameras.

13.3.4 Page 4: Alert Center (/alerts)

The Alert Center provides comprehensive alert management with filtering, batch actions, and detailed investigation tools.

13.3.5 Page 5: Recent Detections (/detections)

Shows all recent detection events with face thumbnails and recognition results.

13.3.6 Page 6: Person Gallery (/persons)

A browsable gallery of all known persons in the system.

13.3.7 Page 7: Unknown Persons Review (/unknowns)

The review queue for unidentifified persons — a critical workflow for building the person database.

13.3.8 Page 8: Person Profile (/persons/{id})

Detailed view of a single person's information, detection history, and management options.

13.3.9 Page 9: Suspicious Activity Timeline (/timeline)

A timeline-based visualization of flagged events for pattern analysis.

13.3.10 Page 10: Search (/search)

Global search across all data types in the system.

13.3.11 Page 11: Watchlists (/watchlists)

Management interface for watchlist categories and their members.

13.3.12 Page 12: AI Vibe Settings (/settings/ai)

The AI Vibe Settings page presents AI configuration as friendly questions rather than technical parameters.

Each setting control:

• Segmented button group (pill-shaped options)
• Selected option highlighted in accent blue
• Brief description below updates on selection
• Current value displayed as badge
• Auto-save (no save button); toast confirms: "Detection Sensitivity updated to High"
• Expand toggle reveals internal numerical values (Admin permission required)

Advanced Mode (Admin only): When expanded, each control shows the internal parameter values:

13.3.13 Page 13: Training Review (/training)

Interface for reviewing AI-suggested face clusters and approving them for model training.

13.3.14 Page 14: System Health (/health)

Real-time system health monitoring dashboard.

13.3.15 Page 15: Notifications Settings (/settings/notifications)

Configuration interface for the notification system.

13.3.16 Page 16: Admin Users (/settings/users)

User management interface for administrators.

13.3.17 Page 17: Camera Management (/settings/cameras)

Configuration interface for camera setup and zone management.

13.3.18 Page 18: Retention & Storage (/settings/storage)

Storage management and retention policy configuration.

13.4 Key User Flows

13.4.1 Flow 1: Daily Operator — Monitor & Respond

┌──────────────────────────────────────────────────────────────────────────────┐
│                  FLOW 1: DAILY OPERATOR (Monitor & Respond)                   │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  STEP 1: LOGIN                                                              │
│  ──────────────                                                              │
│  Enter username → Enter password → MFA code (if enabled)                    │
│  → Redirect to Dashboard                                                     │
│                                                                              │
│  STEP 2: DASHBOARD REVIEW (~30 seconds)                                     │
│  ─────────────────────────────────────                                       │
│  Glance at stat cards:                                                       │
│    ├─ All 8 cameras online? ✓                                               │
│    ├─ Any critical alerts pending? (red badge)                               │
│    ├─ Any unknown persons detected?                                          │
│    └─ System health OK?                                                      │
│                                                                              │
│  If critical alert visible:                                                  │
│    → Click alert card → Go to Alert Center                                  │
│  If no urgent alerts:                                                        │
│    → Click "Live View" in sidebar                                           │
│                                                                              │
│  STEP 3: LIVE CAMERA MONITORING (ongoing)                                   │
│  ─────────────────────────────────────                                       │
│  View 2x4 grid of all cameras                                               │
│  Observe feeds for anomalies                                                │
│                                                                              │
│  When alert toast appears (top-right):                                       │
│    → Toast slides in with sound notification                                 │
│    → Click toast to view alert details                                       │
│                                                                              │
│  STEP 4: ALERT RESPONSE                                                     │
│  ──────────────────                                                          │
│  Click alert toast OR navigate to Alert Center                               │
│  Review alert card:                                                          │
│    ├─ Thumbnail image                                                        │
│    ├─ Camera name, timestamp                                                 │
│    ├─ Alert type (unknown person, watchlist match, etc.)                     │
│    └─ Severity level                                                         │
│                                                                              │
│  Click "View Details" for full information:                                  │
│    ├─ Full-size image / video clip                                           │
│    ├─ AI confidence score                                                    │
│    ├─ Detection metadata (bounding box, zone)                                │
│    └─ Person profile link (if known)                                         │
│                                                                              │
│  DECISION:                                                                   │
│    ├─ False detection → Click "Mark as False Positive"                       │
│    ├─ Legitimate alert → Click "Acknowledge" or "Resolve"                    │
│    ├─ Unknown person → Click "Name This Person"                              │
│    ├─ Needs escalation → Click "Escalate"                                    │
│    └─ Need live view → Click "View Live" to jump to camera                   │
│                                                                              │
│  STEP 5: RETURN TO MONITORING                                               │
│  ────────────────────────────                                                │
│  After handling alert, return to Live View                                   │
│  Continue monitoring cycle                                                   │
│                                                                              │
│  STEP 6: END OF SHIFT                                                       │
│  ──────────────────                                                          │
│  Review unacknowledged alerts (if any)                                       │
│  Check System Health page                                                    │
│  Hand over to next operator (verbal + note any pending issues)               │
│  Click user menu → Logout                                                    │
└──────────────────────────────────────────────────────────────────────────────┘

13.4.2 Flow 2: New Person Onboarding

┌──────────────────────────────────────────────────────────────────────────────┐
│                    FLOW 2: NEW PERSON ONBOARDING                              │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  TRIGGER: System detects unknown person → Alert created → Operator notified │
│                                                                              │
│  STEP 1: REVIEW DETECTION                                                    │
│  ────────────────────                                                        │
│  Navigate to "Recent Detections" via sidebar                                 │
│  Filter: "Unknown" (toggle button)                                           │
│  Click on unknown detection card                                             │
│                                                                              │
│  Detail view shows:                                                          │
│    ├─ Full-size face image                                                   │
│    ├─ Camera: CAM-01 (Entrance)                                              │
│    ├─ Timestamp: 2025-01-16 14:32:15                                        │
│    ├─ Confidence: 87.3%                                                      │
│    └─ AI note: "No matching person found in database"                        │
│                                                                              │
│  STEP 2: NAME THE PERSON                                                     │
│  ────────────────────                                                        │
│  Click "Name This Person" button                                             │
│  Modal dialog appears:                                                       │
│                                                                              │
│    ┌────────────────────────────────────┐                                   │
│    │  Name This Person                   │                                   │
│    │                                     │                                   │
│    │  Face: [thumbnail]                  │                                   │
│    │                                     │                                   │
│    │  Full Name *     [____________]     │                                   │
│    │  Role *          [Employee ▼]       │                                   │
│    │  Department      [____________]     │                                   │
│    │  Employee ID     [____________]     │                                   │
│    │  Notes           [____________]     │                                   │
│    │  Tags            [____________]     │                                   │
│    │                                     │                                   │
│    │  Similar existing persons:          │                                   │
│    │  [No similar persons found]         │                                   │
│    │                                     │                                   │
│    │  [Cancel]  [Save & Create Profile]  │                                   │
│    └────────────────────────────────────┘                                   │
│                                                                              │
│  STEP 3: SIMILARITY CHECK                                                    │
│  ────────────────────                                                        │
│  System searches for similar existing persons                                │
│  If matches found: display side-by-side comparison                           │
│    → Option to merge with existing person instead of creating new            │
│  If no matches: proceed with creation                                        │
│                                                                              │
│  STEP 4: SAVE PROFILE                                                        │
│  ──────────────                                                              │
│  Click "Save & Create Profile"                                               │
│  Toast notification: "Profile created for [Name]"                            │
│  Detection card updates with person name                                     │
│  Person now appears in Person Gallery                                        │
│                                                                              │
│  STEP 5: ADD TRAINING IMAGES (Optional)                                      │
│  ────────────────────────────────────                                        │
│  Navigate to Person Profile                                                  │
│  Click "Upload Reference Photos"                                             │
│  Select additional clear face images                                         │
│  System queues for model retraining                                          │
│  Toast: "3 new training images added. Model will retrain automatically."     │
└──────────────────────────────────────────────────────────────────────────────┘

13.4.3 Flow 3: Unknown Person Review Queue

┌──────────────────────────────────────────────────────────────────────────────┐
│                  FLOW 3: UNKNOWN PERSON REVIEW QUEUE                          │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  STEP 1: OPEN REVIEW QUEUE                                                   │
│  ──────────────────────                                                      │
│  Sidebar → "Unknown Persons Review"                                          │
│  View: Grid of unknown person cluster cards                                  │
│  Header: "23 unknown clusters remaining"                                     │
│                                                                              │
│  STEP 2: SELECT CLUSTER                                                      │
│  ────────────────                                                            │
│  Click on a cluster card to expand                                           │
│  Shows:                                                                      │
│    ├─ Representative face (largest)                                          │
│    ├─ Gallery of all face instances in cluster                               │
│    ├─ Sighting history (camera, time, count)                                 │
│    ├─ AI pattern insight: "Seen 5x at entrance between 08:00-09:00"         │
│    └─ Confidence distribution graph                                          │
│                                                                              │
│  STEP 3: MAKE DECISION                                                       │
│  ────────────────                                                            │
│  Options:                                                                    │
│    ├─ [Name This Person] → Enter details → Create new profile               │
│    ├─ [Merge with Existing] → Search/select person → Confirm merge          │
│    ├─ [Ignore Cluster] → "False detection / not a person" → Remove         │
│    └─ [Mark Reviewed] → "Unsure, keep in queue for later"                   │
│                                                                              │
│  STEP 4: QUEUE UPDATES                                                       │
│  ────────────────                                                            │
│  Processed item removed from queue                                           │
│  Toast confirms action: "Cluster marked as [Name]. 22 remaining."            │
│  Auto-advance to next cluster (optional)                                     │
│  Keyboard shortcut: Right arrow → next cluster                               │
│                                                                              │
│  STEP 5: CONTINUE REVIEW                                                     │
│  ────────────────                                                            │
│  Process all clusters or stop and resume later                               │
│  Queue persists across sessions                                              │
│  New clusters automatically added as detected                                │
└──────────────────────────────────────────────────────────────────────────────┘

13.4.4 Flow 4: AI Settings Adjustment

┌──────────────────────────────────────────────────────────────────────────────┐
│                    FLOW 4: AI SETTINGS ADJUSTMENT                             │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  STEP 1: NAVIGATE TO AI VIBE SETTINGS                                        │
│  ────────────────────────────────────                                        │
│  Sidebar → "AI Vibe Settings" (Sparkles icon)                                │
│  View: Scrollable page with 7 setting sections                               │
│                                                                              │
│  STEP 2: ADJUST DETECTION SENSITIVITY                                        │
│  ────────────────────────────────                                            │
│  Section: "How carefully should the AI watch?"                               │
│  Current: [Relaxed] [Balanced] [High] [Maximum]                              │
│  Change: Click "High"                                                        │
│  Description updates:                                                        │
│    "High: The AI will catch almost everything.                               │
│     Expect more alerts, including some false positives."                     │
│  Toast: "Detection Sensitivity updated to High"                              │
│  Change takes effect immediately                                             │
│                                                                              │
│  STEP 3: ADJUST ALERT STYLE                                                  │
│  ────────────────────                                                        │
│  Section: "When should alerts be sent?"                                      │
│  Current: [Silent] [Digest] [Normal] [Urgent] [Critical]                     │
│  Change: Click "Critical"                                                    │
│  Description updates:                                                        │
│    "Critical: Only truly important events trigger alerts.                    │
│     All other activity is logged but not alerted."                           │
│  Toast: "Alert Style updated to Critical"                                    │
│                                                                              │
│  STEP 4: REVIEW ADVANCED (Admin only)                                        │
│  ────────────────────────────────────                                        │
│  Click "Expand" on Advanced Settings                                         │
│  Shows internal values:                                                      │
│    Detection Sensitivity: High                                               │
│    └─ Confidence Threshold: 0.55                                             │
│    └─ NMS Threshold: 0.40                                                    │
│    └─ Model: yolo11m.onnx                                                    │
│  Admin can directly edit numerical values                                    │
│                                                                              │
│  STEP 5: DONE                                                                │
│  ────────                                                                    │
│  All changes auto-saved                                                      │
│  Return to monitoring — changes effective immediately                        │
└──────────────────────────────────────────────────────────────────────────────┘

13.4.5 Flow 5: Watchlist Alert Configuration

┌──────────────────────────────────────────────────────────────────────────────┐
│                 FLOW 5: WATCHLIST ALERT CONFIGURATION                         │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  STEP 1: NAVIGATE TO WATCHLISTS                                              │
│  ────────────────────────────                                                │
│  Sidebar → "Watchlists"                                                      │
│  View: Grid of existing watchlist cards                                      │
│  Default: VIP, Blacklist, Authorized, Temporary Access                      │
│                                                                              │
│  STEP 2: CREATE NEW WATCHLIST (Optional)                                     │
│  ────────────────────────────────────                                        │
│  Click "+ New Watchlist"                                                     │
│  Modal:                                                                      │
│    Name: [Security Escort Required]                                          │
│    Icon: [🛡️] (icon picker)                                                 │
│    Color: [Orange] (color picker)                                            │
│    Description: [People who require security escort]                         │
│    Click "Create"                                                            │
│  New watchlist card appears in grid                                          │
│                                                                              │
│  STEP 3: ADD MEMBERS                                                         │
│  ────────────────                                                            │
│  Click on watchlist card                                                     │
│  Click "Add from Gallery"                                                    │
│  Search/select persons to add:                                               │
│    [☑] John Doe                                                             │
│    [☑] Jane Smith                                                           │
│    [☐] Bob Johnson (not selected)                                            │
│  Click "Add to Watchlist"                                                    │
│  Toast: "2 persons added to Security Escort Required"                        │
│                                                                              │
│  STEP 4: CONFIGURE ALERTS                                                    │
│  ────────────────────                                                        │
│  Click "Settings" tab on watchlist detail                                    │
│  Configure:                                                                  │
│    Alert Timing:    [☑] Immediate    [☐] Delayed (___ min)                  │
│    Severity:        [☐] Inherit    [☑] Force Critical                       │
│    Notify Groups:   [☑] Security Team    [☐] Management                     │
│    Media:           [☑] Image    [☑] Video                                  │
│    Quiet Hours:     [☐] Respect global    [☑] Always alert                  │
│    Escalation:      [☑] Enable escalation (5/10/20 min)                     │
│  Click "Save"                                                                │
│                                                                              │
│  STEP 5: TEST                                                                │
│  ────────                                                                    │
│  Click "Test Alert" button                                                   │
│  System sends test alert through configured channels                         │
│  Verify: Telegram message received ✓                                         │
│  Verify: WhatsApp message received ✓                                         │
│  Watchlist is now active and monitoring                                      │
└──────────────────────────────────────────────────────────────────────────────┘

13.5 Component Specifications

13.5.1 Camera Feed Component

13.5.2 Alert Card Component

13.5.3 Stat Card Component

13.6 Toast Notification System

Toast behavior:

• Appears in top-right corner
• Stacks up to 5 toasts simultaneously
• Older toasts pushed down when new ones arrive
• Hovering pauses auto-dismiss timer
• Click to dismiss immediately
• Swipe right to dismiss (mobile)

13.7 Modal System

Modal behavior:

• Backdrop click to close (configurable)
• Escape key to close (configurable)
• Focus trap — Tab cycles within modal
• Return focus to trigger element on close
• Body scroll locked when modal open
• Enter key submits primary action (forms)

13.8 Responsive Behavior

13.9 Keyboard Shortcuts

13.10 Animation Guidelines

13.11 Technology Stack

Section 14: Deployment Plan

14.1 Deployment Architecture Overview

The deployment architecture spans two physical environments: AWS cloud for centralized services and an Intel NUC edge gateway at the surveillance site. Both environments are connected via an encrypted WireGuard VPN tunnel. All deployments use containerization (Docker/Kubernetes) with GitOps-based continuous delivery.

┌──────────────────────────────────────────────────────────────────────────────┐
│                         DEPLOYMENT ARCHITECTURE                               │
│                                                                              │
│    ┌────────────────────────────────────────────────────────────────────┐   │
│    │                        AWS CLOUD                                   │   │
│    │                                                                    │   │
│    │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────────────┐ │   │
│    │  │ Route 53 │──▶  ALB     │──▶  EKS     │──▶  App Pods       │ │   │
│    │  │   DNS    │  │ TLS 1.3  │  │ Cluster  │  │  (FastAPI/Next)  │ │   │
│    │  └──────────┘  └──────────┘  └────┬─────┘  └──────────────────┘ │   │
│    │                                    │                              │   │
│    │  ┌──────────┐  ┌──────────┐  ┌────┴─────┐  ┌──────────────────┐ │   │
│    │  │   S3     │  │   RDS    │  │ ElastiCache│  │  MSK Kafka       │ │   │
│    │  │  Media   │  │ Postgres │  │  Redis    │  │  (Event Bus)     │ │   │
│    │  └──────────┘  └──────────┘  └──────────┘  └──────────────────┘ │   │
│    │                                                                    │   │
│    │  ┌──────────────────────────────────────────────────────────────┐ │   │
│    │  │  WireGuard VPN Gateway (EC2)  ←────→  Edge Gateway          │ │   │
│    │  │  UDP 51820                    Tunnel   (Intel NUC, Site)     │ │   │
│    │  └──────────────────────────────────────────────────────────────┘ │   │
│    └────────────────────────────────────────────────────────────────────┘   │
│                                                                              │
│    ┌────────────────────────────────────────────────────────────────────┐   │
│    │                        EDGE SITE                                   │   │
│    │                                                                    │   │
│    │  ┌─────────────────────────────────────────────────────────────┐  │   │
│    │  │              Intel NUC (Ubuntu Server 22.04)                │  │   │
│    │  │                                                             │  │   │
│    │  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │  │   │
│    │  │  │ Video Capture│  │ AI Inference │  │   MinIO      │    │  │   │
│    │  │  │  (RTSP/FFmpeg)│  │ (YOLO/Face) │  │  (Storage)   │    │  │   │
│    │  │  └──────────────┘  └──────────────┘  └──────────────┘    │  │   │
│    │  │                                                             │  │   │
│    │  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │  │   │
│    │  │  │    Redis     │  │  WireGuard   │  │ Node Exporter│    │  │   │
│    │  │  │   (Cache)    │  │   (VPN)      │  │  (Metrics)   │    │  │   │
│    │  │  └──────────────┘  └──────────────┘  └──────────────┘    │  │   │
│    │  │                                                             │  │   │
│    │  └─────────────────────────────────────────────────────────────┘  │   │
│    │                              │                                     │   │
│    │                    ┌─────────┴──────────┐                         │   │
│    │                    │  Camera LAN         │                         │   │
│    │                    │  CP PLUS DVR        │                         │   │
│    │                    │  192.168.29.200:554 │                         │   │
│    │                    │  (8 channels)       │                         │   │
│    │                    └─────────────────────┘                         │   │
│    └────────────────────────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────────────────────────┘

14.2 Cloud Deployment (AWS EKS)

14.2.1 EKS Cluster Configuration

14.2.2 Cloud Service Resources

14.3 Edge Deployment (Intel NUC)

14.3.1 Edge Hardware Specification

14.3.2 Edge Docker Compose Configuration

version: "3.8"

services:
  # RTSP stream capture and frame extraction
  video-capture:
    image: sentinel/surveillance-video-capture:v2.3.1
    restart: unless-stopped
    network_mode: host
    environment:
      - DVR_IP=192.168.29.200
      - DVR_PORT=554
      - NUM_CHANNELS=8
      - FRAME_EXTRACT_FPS=1
      - RECORDING_SEGMENT_SEC=10
      - REDIS_HOST=localhost
      - REDIS_PORT=6379
      - MINIO_ENDPOINT=localhost:9000
    volumes:
      - /data/frames:/app/frames
      - /data/recordings:/app/recordings
      - ./secrets:/run/secrets:ro
    depends_on:
      - redis
      - minio
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 4G
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "5"

  # AI inference service (lightweight edge models)
  ai-inference:
    image: sentinel/surveillance-ai-inference:edge-v2.3.1
    restart: unless-stopped
    runtime: nvidia  # If NVIDIA GPU available; fallback to CPU
    environment:
      - MODEL_PATH=/models
      - REDIS_HOST=localhost
      - REDIS_PORT=6379
      - MINIO_ENDPOINT=localhost:9000
      - INFERENCE_BATCH_SIZE=8
      - CONFIDENCE_THRESHOLD=0.7
      - NMS_THRESHOLD=0.45
    volumes:
      - ./models:/models:ro
      - /data/frames:/app/frames:ro
      - ./secrets:/run/secrets:ro
    depends_on:
      - redis
    deploy:
      resources:
        limits:
          cpus: '6.0'
          memory: 8G
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "5"

  # Local object storage (S3-compatible)
  minio:
    image: minio/minio:RELEASE.2024-latest
    restart: unless-stopped
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - /data/minio:/data
    environment:
      - MINIO_ROOT_USER_FILE=/run/secrets/minio_user
      - MINIO_ROOT_PASSWORD_FILE=/run/secrets/minio_password
    secrets:
      - minio_user
      - minio_password
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1G

  # Local cache and Pub/Sub
  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: >
      redis-server
      --requirepass """
      --appendonly yes
      --maxmemory 512mb
      --maxmemory-policy allkeys-lru
    volumes:
      - redis_data:/data
    ports:
      - "127.0.0.1:6379:6379"
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M

  # WireGuard VPN client
  wireguard:
    image: linuxserver/wireguard:latest
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./wireguard-config:/config
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    deploy:
      resources:
        limits:
          cpus: '0.25'
          memory: 64M

  # Metrics exporter for Prometheus
  node-exporter:
    image: prom/node-exporter:latest
    restart: unless-stopped
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'

volumes:
  redis_data:
    driver: local

secrets:
  minio_user:
    file: ./secrets/minio_user.txt
  minio_password:
    file: ./secrets/minio_password.txt

14.4 Configuration and Environment Variables

14.4.1 Environment Structure

14.4.2 Required Environment Variables

# ─── APPLICATION ───
APP_ENV=production                    # dev | staging | production
APP_NAME="Sentinel AI Surveillance"
APP_VERSION=2.3.1
APP_DEBUG=false
APP_SECRET_KEY=<random-256-bit-key>   # Used for session signing
LOG_LEVEL=INFO                         # DEBUG | INFO | WARNING | ERROR | CRITICAL

# ─── SERVER ───
API_HOST=0.0.0.0
API_PORT=8080
WORKERS=4                              # Uvicorn worker processes
TIMEZONE=Asia/Kolkata

# ─── DATABASE ───
DATABASE_URL=postgresql://user:pass@rds-endpoint:5432/surveillance
DB_POOL_SIZE=20
DB_MAX_OVERFLOW=10
DB_POOL_TIMEOUT=30
DB_ECHO=false                          # Set true for SQL logging (dev only)

# ─── REDIS ───
REDIS_URL=redis://:password@redis-endpoint:6379/0
REDIS_POOL_SIZE=50
REDIS_SOCKET_TIMEOUT=5

# ─── OBJECT STORAGE (S3 or MinIO) ───
STORAGE_TYPE=s3                        # s3 | minio
STORAGE_ENDPOINT=s3.amazonaws.com
STORAGE_BUCKET=sentinel-surveillance-media
STORAGE_REGION=ap-south-1
STORAGE_ACCESS_KEY=<access-key>
STORAGE_SECRET_KEY=<secret-key>
STORAGE_SECURE=true
STORAGE_URL_EXPIRY=300                 # Signed URL expiry in seconds

# ─── DVR / CAMERA CONNECTION ───
DVR_IP=192.168.29.200
DVR_PORT=554
DVR_USERNAME=admin
DVR_PASSWORD=<dvr-password>
DVR_CHANNELS=8
DVR_STREAM_QUALITY=0                   # 0=main (high), 1=sub (low)
DVR_RTSP_TEMPLATE="rtsp://{user}:{pass}@{ip}:{port}/user={user}&password={pass}&channel={ch}&stream={quality}.sdp?"

# ─── AI MODELS ───
MODEL_PATH=/models
HUMAN_DETECTION_MODEL=yolo11m.onnx
FACE_DETECTION_MODEL=scrfd_10g_bnkps.onnx
FACE_RECOGNITION_MODEL=arcface_r100.onnx
CONFIDENCE_THRESHOLD=0.7
NMS_THRESHOLD=0.45
FACE_MATCH_THRESHOLD=0.70
UNKNOWN_CLUSTER_EPS=0.35
UNKNOWN_CLUSTER_MIN_SAMPLES=3

# ─── TELEGRAM NOTIFICATIONS ───
TELEGRAM_ENABLED=true
TELEGRAM_BOT_TOKEN=<bot-token>
TELEGRAM_WEBHOOK_URL=https://api.example.com/webhooks/telegram
TELEGRAM_WEBHOOK_SECRET=<webhook-secret>
TELEGRAM_ADMIN_CHAT_ID=<admin-chat-id>

# ─── WHATSAPP NOTIFICATIONS ───
WHATSAPP_ENABLED=true
WHATSAPP_API_VERSION=v18.0
WHATSAPP_ACCESS_TOKEN=<access-token>
WHATSAPP_PHONE_NUMBER_ID=<phone-number-id>
WHATSAPP_WEBHOOK_VERIFY_TOKEN=<verify-token>
WHATSAPP_BUSINESS_ACCOUNT_ID=<business-account-id>

# ─── VPN ───
VPN_ENABLED=true
VPN_TYPE=wireguard
VPN_ENDPOINT=wg.example.com:51820
VPN_PUBLIC_KEY=<server-public-key>
VPN_PRIVATE_KEY=<client-private-key>
VPN_PRESHARED_KEY=<preshared-key>
VPN_ALLOWED_IPS=10.100.0.0/16
VPN_KEEPALIVE=25

# ─── AUTHENTICATION ───
JWT_SECRET_KEY=<ecdsa-private-key-pem>
JWT_PUBLIC_KEY=<ecdsa-public-key-pem>
JWT_ALGORITHM=ES256
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=15
JWT_REFRESH_TOKEN_EXPIRE_DAYS=7
MFA_REQUIRED_ROLES=super_admin,admin
MFA_ISSUER="Sentinel AI Surveillance"

# ─── MONITORING ───
PROMETHEUS_ENABLED=true
METRICS_PORT=9090
GRAFANA_URL=https://grafana.example.com
SENTRY_DSN=<sentry-dsn>
HEALTH_CHECK_INTERVAL=30

# ─── RETENTION ───
RECORDING_RETENTION_DAYS=90
DETECTION_SNAPSHOT_RETENTION_DAYS=90
EVENT_LOG_RETENTION_DAYS=365
AUDIT_LOG_RETENTION_DAYS=365
TRAINING_DATA_RETENTION_DAYS=365
AUTO_CLEANUP_ENABLED=true
AUTO_CLEANUP_HOUR=3                    # 3:00 AM daily

# ─── SECURITY ───
CORS_ALLOWED_ORIGINS=https://app.example.com,https://staging.example.com
CSP_REPORT_ONLY=false
RATE_LIMIT_DEFAULT=100/minute
RATE_LIMIT_AUTH=10/minute
SESSION_MAX_AGE_HOURS=8
SESSION_IDLE_TIMEOUT_MINUTES=30

14.5 Rollout Stages

14.5.1 Stage 1: Foundation (Weeks 1-4)

Objective: Infrastructure, VPN connectivity, and core data layer operational.

Milestone M1 — Infrastructure Ready (End of Week 4):

• All cloud services deployed and healthy
• VPN tunnel established and stable (< 100ms latency)
• Edge gateway online, all Docker services running
• Database schema deployed with migrations and seed data
• All 8 camera streams reachable from edge
• Basic monitoring and alerting in place

14.5.2 Stage 2: Core AI Pipeline (Weeks 5-8)

Objective: Video ingestion, AI detection, face recognition, and basic API operational.

Milestone M2 — AI Pipeline Operational (End of Week 8):

• All 8 camera streams ingesting at target FPS
• Human detection, face detection, and face recognition operational
• Detection events stored and queryable
• Person matching (known/unknown) working
• Basic REST API serving authenticated requests
• End-to-end: Camera → Detection → Database → API

14.5.3 Stage 3: Application Layer (Weeks 9-12)

Objective: Web dashboard, alerting, notifications, and person management operational.

Milestone M3 — Application Live (End of Week 12):

• Web dashboard accessible with live camera feeds
• Alerts generated and delivered via Telegram and WhatsApp
• Person management (add, view, match, review unknowns) working
• Watchlist alerts functional with correct routing
• Real-time updates via WebSocket
• All RBAC permissions enforced in UI

14.5.4 Stage 4: Intelligence (Weeks 13-16)

Objective: Night mode, training pipeline, self-learning, and advanced features.

Milestone M4 — Intelligence Features Live (End of Week 16):

• Night mode detection operational
• Training pipeline runs and improves models
• Self-learning suggestions appear in review queue
• Privacy modes configurable and effective
• Suspicious activity alerts functional
• Search returns results in acceptable time
• All AI Vibe Settings controls operational

14.5.5 Stage 5: Hardening (Weeks 17-20)

Objective: Security hardening, testing framework, operations readiness, production go-live.

Milestone M5 — Production Go-Live (End of Week 20):

• Security audit complete with all findings addressed
• Self-test framework passing (score >= 85)
• DR tested and verified (RTO < 1 hour, RPO < 15 minutes)
• Operations team trained and runbooks reviewed
• Load test passed at 64-camera target
• 98-item go-live checklist: all items complete
• System stable in production for 72+ hours

14.6 Kubernetes Manifests Overview

14.7 VPN Setup Procedure

14.7.1 Cloud VPN Gateway Setup

#!/bin/bash
# cloud-vpn-setup.sh — Run on cloud VPN EC2 instance

# 1. System preparation
sudo apt update && sudo apt install -y wireguard wireguard-tools iptables-persistent

# 2. Generate WireGuard keys
wg genkey | sudo tee /etc/wireguard/privatekey | wg pubkey | sudo tee /etc/wireguard/publickey

# 3. Create WireGuard configuration
sudo tee /etc/wireguard/wg0.conf << 'EOF'
[Interface]
Address = 10.200.0.1/24
ListenPort = 51820
PrivateKey = <CLOUD_PRIVATE_KEY>
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

# Edge Gateway peer
[Peer]
PublicKey = <EDGE_PUBLIC_KEY>
PresharedKey = <PRESHARED_KEY>
AllowedIPs = 10.200.0.2/32, 192.168.29.0/24
PersistentKeepalive = 25
EOF

# 4. Enable IP forwarding
sudo sysctl -w net.ipv4.ip_forward=1
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf

# 5. Start WireGuard
sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0

# 6. Verify
sudo wg show
ping -c 3 10.200.0.2

14.7.2 Edge VPN Client Setup

#!/bin/bash
# edge-vpn-setup.sh — Run on Intel NUC edge gateway

# 1. Install WireGuard
sudo apt update && sudo apt install -y wireguard wireguard-tools

# 2. Generate keys
wg genkey | sudo tee /etc/wireguard/privatekey | wg pubkey | sudo tee /etc/wireguard/publickey

# 3. Configure
sudo tee /etc/wireguard/wg0.conf << 'EOF'
[Interface]
Address = 10.200.0.2/32
PrivateKey = <EDGE_PRIVATE_KEY>
DNS = 10.100.0.2

[Peer]
PublicKey = <CLOUD_PUBLIC_KEY>
PresharedKey = <PRESHARED_KEY>
Endpoint = <CLOUD_PUBLIC_IP>:51820
AllowedIPs = 10.100.0.0/16, 10.200.0.0/24
PersistentKeepalive = 25
EOF

# 4. Start and enable
sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0

# 5. Verify connectivity
ping -c 3 10.200.0.1        # Cloud VPN gateway
ping -c 3 10.100.0.2        # Cloud DNS/internal service

14.8 Database Initialization

14.8.1 Migration Strategy

Database migrations are managed with Alembic (SQLAlchemy) and executed as Kubernetes init containers before application startup:

initContainers:
  - name: db-migrations
    image: sentinel/surveillance-api:v2.3.1
    command: ["alembic", "upgrade", "head"]
    env:
      - name: DATABASE_URL
        valueFrom:
          secretKeyRef:
            name: db-credentials
            key: url
    resources:
      limits:
        cpu: "500m"
        memory: "256Mi"
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false

Migration Rules:

14.9 SSL Certificate Setup

14.9.1 cert-manager Configuration

# ClusterIssuer for Let's Encrypt production
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
      - http01:
          ingress:
            class: nginx
        selector: {}

---
# Certificate resource
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: sentinel-tls
  namespace: sentinel
spec:
  secretName: sentinel-tls-secret
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - app.example.com
    - api.example.com
    - ws.example.com
  usages:
    - digital signature
    - key encipherment
  privateKey:
    algorithm: ECDSA
    size: 256

Section 15: Testing Plan

15.1 Testing Strategy Overview

The testing strategy encompasses five levels of testing, from isolated unit tests to full system end-to-end validation. The goal is comprehensive coverage of all functional and non-functional requirements with automated execution in CI/CD.

┌──────────────────────────────────────────────────────────────────────────────┐
│                         TESTING PYRAMID                                       │
│                                                                              │
│                        ┌─────────┐                                          │
│                        │  E2E    │  ~20 tests                               │
│                        │ Tests   │  Full system scenarios                   │
│                        ├─────────┤                                          │
│                      ┌─────────────┐                                        │
│                      │ Integration │  ~100 tests                            │
│                      │   Tests     │  Service-to-service                    │
│                      ├─────────────┤                                        │
│                   ┌───────────────────┐                                    │
│                   │    Unit Tests      │  ~300 tests                        │
│                   │  (Components, AI)  │  Isolated functions                  │
│                   └───────────────────┘                                    │
└──────────────────────────────────────────────────────────────────────────────┘

15.2 Unit Testing Strategy

15.3 Integration Testing

15.4 System Testing (End-to-End)

15.5 Load Testing Plan

15.6 Failover Testing

15.7 Security Testing

15.8 AI Pipeline Testing

15.9 Notification Testing

15.10 Test Environments

15.11 CI/CD Pipeline for Testing

┌─────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
│  Push   │──▶│   Lint   │──▶│  Unit    │──▶│  SAST    │──▶│  Build   │
│         │   │ + Format │   │  Tests   │   │ + Scan   │   │  Images  │
└─────────┘   └──────────┘   └──────────┘   └──────────┘   └──────────┘
                                                                   │
                                                                   ▼
┌─────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
│ Deploy  │◀──│   E2E    │◀──│   DAST   │◀──│  Image   │◀──│  Push    │
│Staging  │   │  Tests   │   │  Scan    │   │  Scan    │   │ Registry │
└─────────┘   └──────────┘   └──────────┘   └──────────┘   └──────────┘
      │
      ▼
┌─────────┐
│ Deploy  │ (Manual approval required)
│   Prod  │
└─────────┘

Section 16: Self-Test Framework

16.1 Framework Architecture

The Self-Test Framework is a standalone FastAPI service that continuously validates platform health and readiness through automated test execution.

┌──────────────────────────────────────────────────────────────────────────────┐
│                     SELF-TEST FRAMEWORK ARCHITECTURE                          │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │                    TEST ORCHESTRATOR (FastAPI)                       │   │
│   │                                                                      │   │
│   │   Scheduler        Queue         Executor       Aggregator          │   │
│   │   (cron/APScheduler) │           (asyncio)        │                 │   │
│   │        │            │            │               │                  │   │
│   │   15m health ◄─────┼────────────┼───────────────┤                  │   │
│   │   Daily 3am  ◄─────┼────────────┼───────────────┤                  │   │
│   │   On-demand  ◄─────┼────────────┼───────────────┤                  │   │
│   │                      │            │               │                  │   │
│   │                      ▼            ▼               ▼                  │   │
│   │              ┌─────────────────────────────────────┐                 │   │
│   │              │         Reporter + Storage           │                 │   │
│   │              │  PostgreSQL + S3 (evidence)          │                 │   │
│   │              └─────────────────────────────────────┘                 │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                              │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │                     21 TEST SUITES (170+ CASES)                      │   │
│   │                                                                      │   │
│   │   Infrastructure (TC-01..04)    │   Core AI (TC-05..10)             │   │
│   │   Alerts (TC-11..13)            │   Search (TC-14)                  │   │
│   │   Training (TC-15)              │   Security (TC-16..17)            │   │
│   │   Resilience (TC-18..21)        │                                   │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────────────────────────┘

16.2 Test Suite Catalog (21 Suites)

Total: 21 suites, 170 test cases

16.3 Test Scheduling

16.4 Production Readiness Scoring

Base Score: 100.0

Deductions:
  P0 failure:  -20.0 points each
  P1 failure:  -10.0 points each
  P2 failure:  -5.0 points each
  P3 failure:  -2.0 points each

Minimum score: 0.0
Maximum score: 100.0

16.5 Report Generation

Section 17: Sample Self-Test Report

17.1 Report Header

================================================================================
           SENTINEL AI SURVEILLANCE PLATFORM — SELF-TEST REPORT
================================================================================

Report ID:          STR-20250116-030015
Generated:          2025-01-16 03:00:15 UTC
Environment:        production
Version:            v2.3.1
Triggered By:       Scheduled (Daily 3:00 AM)
Duration:           18 minutes 42 seconds
Overall Status:     GO WITH CAVEATS

17.2 Executive Summary

Priority Breakdown:

17.3 System Metrics at Test Time

17.4 Failed Test Cases

Failure 1: TC-ALT-12-004 — Telegram Media Group Delivery

Description: Verify that media group (multiple images) is delivered correctly via Telegram when an alert contains multiple evidence images.

Expected Result: All 3 images delivered as a media group album within 10 seconds.

Actual Result: Only 2 of 3 images delivered. Third image failed with error: telegram_api_error: Request Entity Too Large (413). Image size after processing: 10.8 MB (exceeds Telegram's 10 MB per-image limit for media groups).

Root Cause: The media processing pipeline resizes images to 1280x720 but does not enforce a hard 10 MB per-image cap for Telegram media groups. The iterative quality reduction loop stops at quality 50 but can still produce files > 10 MB.

Recommended Fix: Add a hard size cap check after image processing. If image exceeds 10 MB after quality reduction to 50%, apply additional compression (reduce dimensions or use WebP format).

Workaround: Single-image delivery mode works correctly. Multi-image alerts temporarily deliver images individually.

Failure 2: TC-RES-20-006 — AI Inference Recovery After Simulated Crash

Description: Verify that the AI inference service recovers and resumes processing within 60 seconds after a simulated process crash.

Expected Result: AI inference pod restarts and resumes processing frames within 60 seconds for all 8 cameras.

Actual Result: Pod restarted successfully (18 seconds), but detection did not resume for Camera 3 and Camera 7. Other 6 cameras resumed within 45 seconds. Root cause: model warm-up process failed due to a race condition in GPU memory allocation during concurrent channel initialization.

Root Cause: All 8 channel processors attempt to load the face recognition model simultaneously. On resource-constrained edge hardware, this causes OOM for channels that lose the initialization race.

Recommended Fix: Implement shared model loading — load each model once and share across all channel processors. Add initialization semaphore.

Workaround: Manual restart of affected channel processors via admin API.

17.5 Trending (Last 14 Days)

17.6 Conclusion and Recommendations

Verdict: GO WITH CAVEATS

The Sentinel AI Surveillance Platform is operational and safe to use. All 42 P0 (Critical) test cases passed, confirming that core surveillance functions are working correctly.

Two P1 (High) priority issues were identified with documented workarounds. Both fixes are scheduled for v2.3.2.

Recommended Actions:

• Address TC-ALT-12-004: Add aggressive compression for Telegram media group images
• Address TC-RES-20-006: Implement shared model loading in AI inference service
• Monitor Telegram multi-image alert delivery metrics (workaround active)
• Monitor AI inference recovery metrics (manual restart documented in runbook)
• Validate both fixes in next daily test run after v2.3.2 deployment

Section 18: Risks and Mitigations

18.1 Risk Register Summary

18.2 Critical Risks Requiring Immediate Action

1. T1 — DVR Disk Full (Score: 20)

   • Action: Emergency disk cleanup within 24 hours
   • Implement automatic rotation at 85% capacity
   • Configure critical alerts at 90%, 95%, 98%
   • Owner: Platform Team | Due: 2025-01-17

2. O1 — Edge Hardware Failure (Score: 15)

   • Action: Procure cold spare device
   • Document hardware replacement runbook
   • Automate configuration restoration from GitOps
   • Owner: Operations Team | Due: 2025-02-01

Section 19: Final Implementation Roadmap

19.1 Five-Phase Implementation (20 Weeks)

19.2 Key Milestones and Deliverables

19.3 Phase Details

Phase 1 (Weeks 1-4): VPC, EKS, RDS, Redis, Kafka, S3, WireGuard VPN, edge gateway OS hardening, Docker setup, database schema with migrations, monitoring stack (Prometheus, Grafana).

Phase 2 (Weeks 5-8): RTSP capture service, YOLO human detection, SCRFD face detection, ArcFace face recognition, embedding storage with pgvector, person matching logic, FastAPI core, authentication.

Phase 3 (Weeks 9-12): Next.js frontend, design system, dashboard, live camera view (HLS), alert engine with rules, Telegram Bot API integration, WhatsApp Business API integration, person gallery and profile, unknown review queue, watchlists, WebSocket real-time updates.

Phase 4 (Weeks 13-16): Night mode AI model, AI Vibe Settings page, training pipeline with model versioning, self-learning service for unknown clusters, privacy mode with face blurring, suspicious activity detection, search service (face + text), system health dashboard.

Phase 5 (Weeks 17-20): Penetration testing, SAST/DAST, self-test framework (21 suites), backup/DR setup, incident response runbooks, load testing (8-64 cameras), performance tuning, operations training, go-live checklist (98 items), production cutover, 72-hour stability monitoring.

19.4 Resource Allocation

Section 20: Final Production-Readiness Summary

20.1 System at a Glance

20.2 Readiness Checklist Summary

20.3 Estimated Timeline

Appendices

Appendix A: Cross-Reference to Specialist Documents