idps-escape

SONAR documentation

Comprehensive documentation for SONAR (SIEM-Oriented Neural Anomaly Recognition) - the scenario-based anomaly detection system for IDPS-ESCAPE.

⚠️ All commands should be run from project root (/home/alab/soar)

💻 For Python API reference: See sonar/README.md for developer quickstart and programmatic usage examples.

Documentation overview

Document	Purpose	Audience
setup-guide.md	Installation, configuration, CLI usage, debug mode	Everyone
scenario-guide.md	Complete scenario system guide with examples	Users & Developers
data-injection-guide.md	Inject synthetic data for fast testing	Testers & Developers
data-shipping-guide.md	Ship anomalies to Wazuh data streams (production feature)	Operators & DevOps
model-naming-guide.md	Model naming and versioning strategies	Users & Developers
troubleshooting.md	Error solutions and diagnostics	Everyone
architecture.md	System design, patterns, and principles	Developers & Architects
uml-diagrams.md	Visual system design diagrams	Developers & Architects

Getting started

New users (5 minutes)

# 1. Install (from project root)
cd /home/alab/soar
poetry install --with sonar

# 2. Run a scenario (from project root)
poetry run sonar scenario --use-case sonar/scenarios/brute_force_detection.yaml --debug

Read: setup-guide.md

Understanding scenarios (15 minutes)

Review built-in scenarios in sonar/scenarios/ (from project root):
- brute_force_detection.yaml - SSH/authentication attacks
- lateral_movement_detection.yaml - Network traversal attacks
- privilege_escalation_detection.yaml - Privilege abuse
- linux_resource_monitoring.yaml - Resource utilization anomalies
Read scenario structure in scenario-guide.md
Create custom scenario following templates

Troubleshooting issues

Check troubleshooting.md for:

Connection errors
Training/detection issues
Data quality problems
Performance optimization

Architecture deep dive

For developers extending the system:

Read architecture.md - design principles and module structure
Review uml-diagrams.md - class and sequence diagrams
Examine source code in ../ with architecture context

Quick reference by task

Task	Document	Section
Install SONAR	setup-guide.md	Installation
Configure Wazuh connection	setup-guide.md	Configuration
Run first scenario	scenario-guide.md	Overview → Example
Test with synthetic data	data-injection-guide.md	Quick start
Enable data shipping (production)	data-shipping-guide.md	Quick start
Integrate with RADAR	data-shipping-guide.md	Integration with RADAR
View system diagrams	uml-diagrams.md	All diagrams

Key concepts

Scenario-based approach

SONAR uses YAML scenarios to define anomaly detection workflows:

name: "Brute Force Detection"
training:
  lookback_hours: 168  # 1 week baseline
  numeric_fields: ["rule.level"]
detection:
  mode: "batch"
  threshold: 0.7

See scenario-guide.md for complete details.

Flexible execution

Scenarios automatically execute based on sections present:

Training + Detection: Full workflow (train model, then detect)
Training only: Build baseline model without detection
Detection only: Use existing model for investigation

See scenario-guide.md

Debug mode

Test without Wazuh infrastructure:

poetry run sonar scenario --use-case scenario.yaml --debug

Uses local JSON test data for development and CI/CD.

See setup-guide.md

Configuration separation

Config files (sonar/default_config.yaml): Infrastructure (Wazuh connection, paths)
Scenario files (sonar/scenarios/*.yaml): Detection logic (features, thresholds)

See setup-guide.md

System architecture

Conceptual view (high-level components):

CLI Commands (train, detect, scenario, check)
    ↓
Configuration (YAML → Python dataclasses)
    ↓
Data Ingestion (WazuhIndexerClient | LocalDataProvider)
    ↓
Feature Engineering (WazuhFeatureBuilder)
    ↓
ML Engine (MVADModelEngine - Microsoft MVAD)
    ↓
Post-processing (MVADPostProcessor)
    ↓
Storage (Wazuh Indexer / OpenSearch)

Details: architecture.md

Built-in scenarios

Scenario	Use Case	File (from project root)
Brute Force Detection	Authentication attacks	`sonar/scenarios/brute_force_detection.yaml`
Lateral Movement	Cross-host authentication	`sonar/scenarios/lateral_movement_detection.yaml`
Privilege Escalation	Unusual sudo/su usage	`sonar/scenarios/privilege_escalation_detection.yaml`
Resource Monitoring	CPU/memory anomalies	`sonar/scenarios/linux_resource_monitoring.yaml`

See scenario-guide.md

Testing

# All tests
poetry run pytest tests/

# Specific component
poetry run pytest tests/engine_test.py

# With coverage
poetry run pytest --cov=sonar

Test files in ../../tests/

Integration

Wazuh

Input: Reads from wazuh-alerts-* indices
Output: Writes anomalies to wazuh-anomalies-mvad
Authentication: Basic auth (configurable)

RADAR

SONAR anomalies trigger RADAR automated responses:

RADAR monitors anomaly index
High-confidence anomalies trigger playbooks
Integration via shared Wazuh Indexer

Details: setup-guide.md

Project structure

Relative to project root (/home/alab/soar/):

sonar/
├── README.md                  # SONAR overview
├── scenarios/                 # Built-in detection scenarios
│   ├── brute_force_detection.yaml
│   ├── lateral_movement_detection.yaml
│   ├── privilege_escalation_detection.yaml
│   └── linux_resource_monitoring.yaml
├── test_data/                 # Test datasets for debug mode
│   ├── synthetic_alerts/
│   ├── generated_scenarios/
│   └── resource_monitoring/
├── models/                    # Trained model storage
├── cli.py                     # Command-line interface
├── config.py                  # Configuration system
├── scenario.py                # Scenario loader
├── engine.py                  # MVAD model wrapper
├── features.py                # Feature engineering
├── pipeline.py                # Post-processing
├── wazuh_client.py            # Wazuh API client
└── local_data_provider.py     # Debug mode data provider

Contributing

When extending SONAR:

Follow existing patterns in architecture.md
Add tests for new features
Update relevant documentation
Test with debug mode before production

Documentation conventions

Lowercase filenames: setup-guide.md, not Setup-Guide.md
Sentence case headings: “## Getting started”, not “## Getting Started”
Code examples: Include complete, runnable examples
Internal links: Use relative paths: [file](./file.md)

Documentation standards

All documentation in this folder follows these standards:

Markdown format (.md files)
Cross-linked - Files reference each other with relative paths
Versioned - Last updated dates in file headers (when applicable)
Indexed - This README provides central navigation
Verified - All links tested and working

Link format

Same folder: [File.md](./File.md)
Parent folder: [file.md](../file.md)
Root level: [file.md](../../../file.md)

Quick reference

Run tests

# From project root (/home/alab/soar)
poetry run pytest tests/sonartests/ -v

Execute scenario

# From project root (/home/alab/soar)
poetry run sonar scenario --use-case sonar/scenarios/brute_force_detection.yaml

View documentation

# From project root (/home/alab/soar)
cat docs/manual/sonar_docs/setup-guide.md
cat docs/manual/sonar_docs/scenario-guide.md
cat docs/manual/sonar_docs/architecture.md

Support

For help with specific topics:

Getting started: setup-guide.md
Creating scenarios: scenario-guide.md
Testing and debugging: data-injection-guide.md
Production deployment: data-shipping-guide.md
Errors and issues: troubleshooting.md
System design: architecture.md
Python API: sonar/README.md

This site is open source. Improve this page.