idps-escape

RADAR Architecture Documentation

Introduction

RADAR (Risk-aware Anomaly Detection-based Automated Response) is a security orchestration subsystem within IDPS-ESCAPE that integrates anomaly detection with automated response capabilities. The architecture combines signature-based detection with machine learning-based anomaly detection using Random Cut Forest (RCF) algorithms, orchestrating automated security responses through SOAR playbooks.

Key Capabilities

Core Technologies

Design Principles

Modularity and extensibility

RADAR is designed with a modular architecture where each scenario can be independently developed, deployed, and maintained. New scenarios can be added by creating the necessary components (decoders, rules, active responses, detectors) without modifying the core framework.

Idempotency and repeatability

Through Ansible automation, RADAR ensures that deployments are idempotent—running the same playbook multiple times produces the same result. This allows for safe re-deployment and updates without side effects.

Separation of concerns

The architecture separates:

This separation enables independent scaling, testing, and evolution of each layer.

Hybrid Detection Strategy

RADAR implements a defense-in-depth approach combining:

  1. Signature-based detection: Fast, deterministic rule matching for known threats
  2. Behavior-based detection: ML model detecting novel or sophisticated attacks

This hybrid approach provides resilience against adversarial interference and reduces false positives.

Configuration-Driven Deployment

All scenario specifications (index patterns, features, thresholds, detector parameters) are centralized in config.yaml, enabling declarative infrastructure and reducing manual configuration errors.

System Architecture

High-Level Component Overview

RADAR consists of six primary components working together in an orchestrated pipeline:

Component Role Key Functions
Wazuh Agents Endpoint monitoring Log collection, enrichment, response execution
RADAR Helper Data enrichment Scenario-specific log enhancement
Wazuh Manager Central control Log processing, rule evaluation, response coordination
Opensearch/Wazuh Indexer Data platform Storage, indexing, anomaly detection (RCF models)
Webhook Endpoint Alert routing Notification handling, log writing
RADAR Controller Orchestration Deployment automation, scenario management

Component Diagram

RADAR Component Architecture

Figure 1: RADAR component architecture showing the six primary components and their interactions

Module Structure

Wazuh Agent Module

The Wazuh Agent module runs on monitored endpoints and serves as the data collection and response execution layer.

Logs Collection

RADAR Helper

The RADAR Helper is a scenario-specific enrichment component that augments raw logs with contextual information before transmission to the manager.

Purpose: Enrich logs with scenario-relevant data that aids detection

Functions:

Implementation: Python script deployed to agent endpoints

Active Response

Wazuh Manager Module

The Wazuh Manager is the Wazuh central control plane, it processes incoming logs and orchestrates responses.

Decoders

Purpose: Parse and normalize log data into structured fields

Format: XML-based decoder definitions

Function: Extract relevant fields from enriched logs for rule matching

Rules

Purpose: Define detection logic for security events

Function: Evaluate decoded logs, match with conditions and trigger alerts

Active Response

Purpose: Act on incidents and anomalies

Function:

Monitors

Purpose: Continuously evaluate anomaly detector outputs and trigger webhooks

Configuration: Threshold-based evaluation (anomaly score > threshold and confidence > threshold)

Function:

Webhook Notifications

Purpose: Send anomaly alerts to webhook endpoint for processing

Protocol: HTTP POST with JSON payload

Opensearch Module

Opensearch provides the data platform for storage, search, and ML-based anomaly detection.

Indexer (Wazuh Indexer)

Purpose: Centralized data storage and retrieval

Technology: Opensearch with Wazuh indexer integration

Key Functions:

Index Patterns: Scenario-specific (e.g., wazuh-alerts-4.x-*, custom-suspicious-login-*)

Anomaly Detectors

Purpose: Machine learning-based behavioral anomaly detection

Algorithm: Robust Random Cut Forest (RRCF)

Key Features:

Detection Process:

  1. Query index data at configured intervals
  2. Extract feature values per entity (if slicing enabled)
  3. Compute anomaly scores using RCF algorithm
  4. Store results with timestamps and scores
  5. Monitor evaluates scores against thresholds

Webhook Endpoint Module

The Webhook Endpoint receives anomaly notifications and bridges Opensearch monitors to Wazuh’s rule engine.

HTTP Webhook Service

Purpose: Receive and process anomaly alerts from Opensearch monitors

Implementation: Custom HTTP service (Python/Flask)

Endpoint: POST /notify

Function:

  1. Receive webhook POST request with anomaly data
  2. Parse and validate notification payload
  3. Format alert for Wazuh rule processing
  4. Write alert to monitored log file (e.g., /var/log/ad_alerts.log)
  5. Wazuh Manager detects new log entry and triggers rules

RADAR Controller/Orchestrator Module

The orchestration layer automates deployment, configuration, and lifecycle management.

Key Scripts

build-radar.sh

run-radar.sh

stop-radar.sh

Core components

Anomaly Detector module

The anomaly_detector/ folder contains Python modules that interface with the OpenSearch Anomaly Detection plugin API. These scripts are executed inside the radar-cli Docker container during run-radar.sh.

detector.py

Creates and starts OpenSearch anomaly detectors based on scenario configuration.

Key functions:

Detector specification includes:

Usage: python detector.py <scenario_name> → outputs detector ID

monitor.py

Creates OpenSearch monitors that evaluate detector results and trigger webhooks when anomalies exceed thresholds.

Key functions:

Monitor configuration includes:

Note: The monitor_interval parameter is optional in scenario configurations. If not specified, the monitor uses the same interval as the detector (detector_interval). See monitor.py implementation: int(scn.get("monitor_interval", scn.get("detector_interval", 5))).

Usage: python monitor.py <scenario_name> <detector_id> → outputs monitor ID

webhook.py

Manages OpenSearch notification destinations (webhooks) for monitor alerts.

Key functions:

Usage: Called internally by monitor.py to ensure webhook destination exists before creating monitor.

RADAR Helper

The RADAR Helper is a log enrichment service that runs on Wazuh agents. It tails authentication logs, enriches them with geographic and behavioral context, and writes to a separate log file for Wazuh to ingest.

radar-helper.py

A multi-threaded Python daemon that processes authentication logs in real-time.

Core classes:

Enrichment fields added to each log line:

Field Description
outcome success or failure
asn Autonomous System Number from MaxMind
asn_placeholder_flag true if ASN lookup failed
country ISO country code
region State/province name
city City name
geo_velocity_kmh Calculated travel speed since last login
country_change_i 1 if country changed from previous login
asn_novelty_i 1 if this ASN is new for this user (90-day window)

Key algorithms:

Dependencies:

Output: Writes enriched logs to /var/log/suspicious_login.log

Configuration Files

config.yaml

scenarios/active_responses/ar.yaml

inventory.yaml

.env

volumes.yml

Docker Compose Files

RADAR uses multiple Docker Compose files to provide modular, composable container orchestration. These files can be combined using Docker Compose’s -f flag to build different deployment configurations.

docker-compose.core.yml

Defines the core Wazuh stack:

Service Image Ports Purpose
wazuh.indexer wazuh/wazuh-indexer:4.14.1 9200 OpenSearch-based data storage and anomaly detection engine
wazuh.manager wazuh/wazuh-manager:4.14.1 1514, 1515, 514/udp, 55000 Central log processing, rule evaluation, agent management
wazuh.dashboard Custom build 443→5601 Web UI for Wazuh and OpenSearch (includes AD plugin UI)

Key configuration:

It does not build if own Wazuh already exists (--manager-exists true)

docker-compose.agents.yml

Defines containerized Wazuh agents for local/demo deployments:

Service Container Purpose
agent.insider agent.insider Insider threat scenario agent
agent.ddos agent.ddos DDoS detection agent (exposes port 8800)
agent.malcom agent.malcom Malware communication detection agent
agent.suspicious agent.suspicious Suspicious login agent (Keycloak on port 8080)
agent.geoip agent.geoip GeoIP detection agent
agent.logvolume agent.logvolume Log volume monitoring agent

Each agent:

These agent containers are for lab usage. It does not build if own agent exists, which are defined in inventory.yaml under wazuh_agents_ssh and build-radar.sh is run with key --agent remote.

docker-compose.webhook.yml

Defines the webhook endpoint service for ML-based detection:

Service Container Ports Purpose
webhook ad-webhook 8080 Receives anomaly alerts from OpenSearch monitors

Key configuration:

Dockerfile.radar-cli

Defines the radar-cli container used by run-radar.sh:

Supporting Tools

radar-cli

Docker Compose

Ansible

Flowintel

Data Flow

Overall Data Flow pipeline

Wazuh Agent (Logs) → RADAR Helper (Enrichment) → Wazuh Manager (Decode) → 
    ↓
[Signature-Based Path]
    Wazuh Manager (Rules) → Active Response
    
[Behavior-Based Path]
    Wazuh Indexer → Anomaly Detector → Monitor → Webhook → 
    Wazuh Manager (Rules) → Active Response

Signature-Based Detection Flow

sequenceDiagram
    participant Agent as Wazuh Agent
    participant Helper as RADAR Helper
    participant Manager as Wazuh Manager
    participant Rules as Rules Engine
    participant AR as Active Response

    Agent->>Helper: Raw log event
    Helper->>Helper: Enrich with GeoIP, ASN, velocity
    Helper->>Manager: Enriched log
    Manager->>Manager: Decode log fields
    Manager->>Rules: Evaluate rules
    
    alt Rule matches (e.g., non-whitelist country)
        Rules->>AR: Trigger alert
        AR->>Agent: Execute response
        AR->>AR: Log action
    else No match
        Rules->>Rules: Continue monitoring
    end

Step 1: Log Collection and Enrichment

  1. Wazuh Agent monitors endpoint and collects raw logs
  2. RADAR Helper enriches logs with contextual data
  3. Enriched logs sent to Wazuh Manager

Step 2: Parsing

  1. Manager receives enriched logs
  2. Decoders parse logs into structured fields
  3. Extracted fields made available to rule engine

Step 3: Rule Matching

  1. Rules evaluate decoded fields against conditions
  2. Pattern matching, thresholds, and correlations applied
  3. Matching rules generate alerts with severity levels

Step 4: Active Response

  1. Alerts trigger active response based on severity
  2. Manager sends response command to agent
  3. Agent executes action (block, notify, isolate)
  4. Action logged to active-responses.log

Behavior-Based Detection Flow

sequenceDiagram
    participant Agent as Wazuh Agent
    participant Helper as RADAR Helper
    participant Manager as Wazuh Manager
    participant Index as OpenSearch Index
    participant Detector as Anomaly Detector
    participant Monitor as Monitor
    participant Webhook as Webhook
    participant AR as Active Response

    Agent->>Helper: Raw log event
    Helper->>Helper: Enrich with features
    Helper->>Manager: Enriched log
    Manager->>Manager: Decode log
    Manager->>Index: Store in scenario index
    
    loop Every detection_interval
        Detector->>Index: Query feature data
        Detector->>Detector: Compute RCF anomaly score
        Detector->>Detector: Store results
    end
    
    loop Every monitor_interval
        Monitor->>Detector: Check anomaly scores
        alt Score > threshold AND confidence > threshold
            Monitor->>Webhook: HTTP POST alert
            Webhook->>Webhook: Write to log file
            Webhook->>Manager: Log entry detected
            Manager->>AR: Trigger response
        end
    end

Step 1: Log Collection and Enrichment

  1. Wazuh Agent monitors endpoint and collects raw logs
  2. RADAR Helper enriches logs with scenario-specific features
  3. Enriched logs sent to Wazuh Manager

Step 2: Indexing

  1. Manager processes logs through decoders
  2. Logs forwarded to Wazuh Indexer (OpenSearch)
  3. Data stored in scenario-specific indices

Step 3: Anomaly Detection

  1. Anomaly Detector queries index at configured intervals
  2. Extracts feature values
  3. RCF algorithm computes anomaly scores
  4. Scores stored with entity identifiers and timestamps

Step 4: Monitoring and Alerting

  1. Monitor polls detector results
  2. Evaluates anomaly score and confidence against threshold
  3. When score exceeds threshold, sends webhook notification

Step 5: Webhook Processing

  1. Webhook endpoint receives POST request
  2. Parses anomaly data
  3. Formats alert message
  4. Writes alert to monitored log file

Step 6: Rule Triggering

  1. Wazuh Manager detects new log entry
  2. Decoder parses webhook alert
  3. Rule matches on anomaly alert pattern

Step 7: Active Response

  1. Alert triggers active response based on anomaly severity
  2. Manager executes action
  3. Action logged for audit

Configuration System

Centralized Configuration (config.yaml)

The config.yaml file serves as the single source of truth for all scenario configurations.

Key Sections:

Risk-Aware Active Response Configuration

The ar.yaml file configures the risk-based active response system for each scenario. This file is located at the path specified by the AR_RISK_CONFIG environment variable in .env (default: ar.yaml in the Wazuh manager’s active response directory).

Key Configuration Parameters:

Parameter Description Example Values
ad.rule_ids Rule IDs from ML-based anomaly detection ["100021", "100309"]
signature.rule_ids Rule IDs from signature-based detection ["210012", "210013"]
w_ad Weight for anomaly detection in risk score (0.0-1.0) 0.3 for 30% weight
w_sig Weight for signature-based detection in risk score (0.0-1.0) 0.4 for 40% weight
w_cti Weight for cyber threat intelligence in risk score (0.0-1.0) 0.3 for 30% weight
delta_ad_minutes Time window for AD alerts correlation (minutes) 10
delta_signature_minutes Time window for signature alerts correlation (minutes) 1
signature_impact Impact score for signature detections (0.0-1.0) 0.7
signature_likelihood Likelihood score for signature detections (0.0-1.0) or weighted rules 0.8 or rule-specific weights
risk_threshold Minimum risk score to trigger response (0.0-1.0) 0.51
tiers.tier1_max Maximum risk score for Tier 1 (low risk) 0.33
tiers.tier2_max Maximum risk score for Tier 2 (medium risk) 0.66
mitigations List of active response actions to execute ["firewall-drop", "lock_user_linux"]
create_case Whether to create a case in FlowIntel true or false
allow_mitigation Whether to execute automated mitigations true or false

Environment Variables (.env)

Connection details and credentials stored securely:

OS_URL=https://OS_IP:9200
OS_USER=admin
OS_PASS=SecretPassword
WAZUH_API_URL=https://WAZUH_IP:55000
WAZUH_AGENT_VERSION=4.14.1-1
WEBHOOK_URL=http://WEBHOOK_IP:8080/notify
AR_RISK_CONFIG=/var/ossec/active-response/bin/ar.yaml
AR_LOG_FILE=/var/ossec/logs/active-responses.log
FLOWINTEL_BASE_URL=http://FLOWINTEL_IP:7006/api
FLOWINTEL_API_KEY=API_KEY

Ansible Inventory (inventory.yaml)

Remote host definitions for distributed deployments:

all:
  children:
    wazuh_manager_ssh:
      hosts:
        manager-node:
          ansible_host: 192.168.5.10
          ansible_user: linuxuser
    
    wazuh_agents_ssh:
      hosts:
        agent-node-1:
          ansible_host: 192.168.5.20
          ansible_user: linuxuser
        agent-node-2:
          ansible_host: 192.168.5.21
          ansible_user: linuxuser

Ansible Vault (host_vars/)

Encrypted credentials for remote access:

# Create vault for host
ansible-vault create host_vars/edge.vm.yml

# Content (encrypted):
ansible_become_password: sudo_password_here

Scenario-Based Execution

Available Scenarios

RADAR provides pre-configured scenarios for common security use cases:

Signature-Based Scenarios (Ready for Production)

  1. Non-whitelist GeoIP Detection
    • Detects logins from non-approved countries
    • Response: user notification
  2. Suspicious Login (Signature)
    • Detects brute force, impossible travel patterns
    • Response: user notification

Behavior-Based Scenarios (ML/RCF)

Production-Ready (v0.5+)

  1. Log Volume Growth Detection
    • Detects unusual spikes in log generation
    • Use case: DDoS, malware outbreak detection
    • Response: user notification

Demo/Development (Requires Adaptation) These are currently under radar/archives folder.

  1. Insider Threat
    • Detects anomalous user behavior (data exfiltration, privilege abuse)
    • Response: user lock, logging
  2. Suspicious Login (Behavior)
    • Learns user login patterns, detects deviations
    • Response: user block, logging, firewall drop
  3. DDoS Detection
    • Identifies distributed denial of service patterns
    • Response: firewall drop, rate limiting, logging
  4. C2 Malware Communication
    • Detects command-and-control traffic patterns
    • Response: firewall drop, process termination, logging
This site is open source. Improve this page.