1 Start ADBox interactive console TST-001

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • 14 GB storage on the machine
  • git version 2.44.0

Test steps

  1. Clone the ADBox repository. git clone https://github.com/AbstractionsLab/idps-escape.git
  2. Change the working directory to the cloned folder containing the all the files along with the Dockerfile. cd siem-mtad-gat
  3. Build the image.
docker build -t siem-mtad-gat:v0.1.0 .
  1. Run the container by executing the bash file containing the run commands. a. Make it executable. chmod +x adbox.sh b. And execute it. ./adbox.sh

Expected outcome

  1. Copy of siem-mtad-gat folder in local folder
  2. In the list of docker imageges (docker images) it should appear
REPOSITORY                      TAG       IMAGE ID       CREATED         SIZE
siem-mtad-gat                   v0.1.0    ...   ...            ...

4b. ADBox interactive console should be opened

Child links: TRA-001

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 1
test_data see referenced files

2 Installation of ADBox via pipx TST-002

Preconditions and setup actions

  • Python 3.10.12
  • git version 2.44.0

Test steps

  1. Install pipx
python3 -m pip install pipx
python3 -m pipx ensurepath
  1. Unpack the siem-mtad-gat.zip distribution
  2. Change working directory to the unpacked folder (e.g.,cd siem-mtad-gat) containing the wheel distribution file (.whl), and use pipx to install SIEM-MTAD-GAT
pipx install ./siem_mtad_gat-0.1.0-py3-none-any.whl
  1. And then launch it by: adbox

Expected outcome

  1. pipx successufully installed
  2. creation of 3.
  3. ADBox interactive console should be opened.

Child links: TRA-002

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

3 Install ADBox as dev container TST-003

Preconditions and setup actions

  • Docker engine version 26.1.3
  • Docker desktop version 1.0.35
  • Visual Studio Code version 1.83.1 (system setup)
  • Dev Containers extension for VS Code by Microsoft version v0.315.1
  • at least 26GB storage on the machine

Test steps

  1. Clone this repository: git clone https://github.com/AbstractionsLab/idps-escape.git
  2. Start Docker Desktop if not already running.
  3. Open the project folder in VS Code.
  4. Select the "Reopen in Container" option in the notification that pops up in VS Code.
  5. Open a terminal in VS Code and run poetry install in the container to install all dependencies.
  6. And then run the ADBox using its entrypoint.
 poetry run adbox

Expected outcome

  1. repository creation
  2. Container created and terminal open
  3. Poetry install terminatwe with Installing the current project: siem-mtad-gat (0.1.0)
  4. ADBox interactive console should be opened.

Child links: TRA-003, TRA-004, TRA-020

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

4 Run ADBox console TST-004

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script without any parameters. ./adbox.sh
  2. Press <1>, to select train.
  3. Press <n>, to train using default date.
  4. Press <n>, to name the trained detector with a default name.
  5. Press <n>, to train using default configurations.

Expected outcome

  • ADBox starts with “Running AD driver in console mode.”
  • The training should run for 30 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.

Child links: TRA-005

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

5 Run ADBox in default mode with a Wazuh connection TST-005

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with default parameter parameters. ./adbox.sh -df

Expected outcome

  • ADBox starts with “Running AD driver in default mode.”
  • The training should run for 30 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-006

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 3
test_data see referenced files

6 Run ADBox in default mode without a Wazuh connection TST-006

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with default parameter parameters. ./adbox.sh -df

Expected outcome

  • ADBox starts with “Running AD driver in default mode.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-07.json”.
  • The training should run for 30 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “The file '/home/root/siem-mtad-gat/siem_mtad_gat/assets/data/predict/sample-alerts-prediction-2024-07-22' does not exist, returning all default data.”, depending upon the date it was run.
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-007, TRA-019

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

7 ADBox use case 1 with a Wazuh connection TST-007

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named siem-mtad-gat-container should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.
  • Wazuh configured to monitor linux resource utilization. (Monitoring Linux resource usage with Wazuh)

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 1 parameters. ./adbox.sh -uc 1

Expected outcome

  • ADBox starts with Running AD driver with user configuration uc_1.yaml.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in real-time mode with interval 1 (min).”
  • Prediction response should be seen on the output console after every 1 minute.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-008

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

8 ADBox use case 1 without a Wazuh connection TST-008

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 1 parameters.
 ./adbox.sh -uc 1

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_1.yaml.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-07.json”.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in real-time mode with interval 1 (min).”
  • Output screen should show the following messages:
  • “Could not establish a connection with OpenSearch.”
  • “Prediction in run_mode.REALTIME requires a connection with OpenSearch.”
  • “No data found for given input.”
  • And the application should exit.

Child links: TRA-009, TRA-018

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

9 ADBox use case 2 with a Wazuh connection TST-009

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 2 parameters. ./adbox.sh -uc 2

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_2.yaml.”
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-010

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

10 ADBox use case 2 without a Wazuh connection TST-010

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 2 parameters. ./adbox.sh -uc 2

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_2.yaml.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-07”.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “The file '/home/root/siem-mtad-gat/siem_mtad_gat/assets/data/predict/sample-alerts-predict-2024-07-26' does not exist, returning all default data.”, depending upon the date it was run.
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-017

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

11 ADBox use case 3 with a Wazuh connection TST-011

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 3 parameters.
 ./adbox.sh -uc 3

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_3.yaml.”
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in batch mode with batch interval 5 (min).”
  • Prediction response should be seen on the output console after every 5 minutes.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-011

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

12 ADBox use case 3 without a Wazuh connection TST-012

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 3 parameters.
 ./adbox.sh -uc 3

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_3.yaml.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-07.json”.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in batch mode with batch interval 5 (min).”
  • Output screen should show the following messages:
    • “Could not establish a connection with OpenSearch.”
    • “Prediction in run_mode.BATCH requires a connection with OpenSearch.”
    • “No data found for given input.”
    • And the application should exit.

Child links: TRA-016

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

13 ADBox use case 4 with a Wazuh connection TST-013

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.
  • Wazuh agent configured to read Suricata logs.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 4 parameters.
./adbox.sh -uc 4

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_4.yaml.”
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-012

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

14 ADBox use case 4 without a Wazuh connection TST-014

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 4 parameters.
./adbox.sh -uc 4

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_4.yaml.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-03.json”.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “The file '/home/root/siem-mtad-gat/siem_mtad_gat/assets/data/predict/wazuh-alerts-*.*-2024.07.22.json' does not exist, returning all default data.”, depending upon the date it was run.
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-015

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

15 ADBox use case 5 with a Wazuh connection TST-015

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • An instance of a Wazuh distribution should be running.
  • Wazuh indexer RESTful API should be listening on port 9200.
  • Wazuh agent configured to read Suricata logs.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 5 parameters. ./adbox.sh -uc 5

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_5.yaml.”
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-013

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files

16 ADBox use case 5 without a Wazuh connection TST-016

Preconditions and setup actions

  • Docker version 26.0.0
  • User with root privileges
  • Built image siem-mtad-gat:v0.1.0
  • No existing container named “siem-mtad-gat-container” should be running.
  • No instance of Wazuh distribution running.

Test steps

  1. Run the container “siem-mtad-gat-container” by executing the adbox script with use case 5 parameters.
./adbox.sh -uc 5

Expected outcome

  • ADBox starts with “Running AD driver with user configuration uc_5.yaml.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “Returning data from file /home/root/siem-mtad-gat/siem_mtad_gat/assets/data/train/sample-alerts-train-2024-03.json”.
  • The training should run for 10 epochs.
  • Train response should be seen on the console.
  • Training outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}” folder.
  • Prediction starts after training with message “Predicting in historical mode.”
  • Output screen should show message “Could not establish a connection with OpenSearch.”
  • And collect training data from default file with message “The file '/home/root/siem-mtad-gat/siem_mtad_gat/assets/data/predict/wazuh-alerts-*.*-2024.07.22.json' does not exist, returning all default data.”, depending upon the date it was run.
  • Prediction response should be seen on the output console.
  • Prediction outputs and artifacts should be available in the “/home/root/siem-mtad-gat/siem_mtad_gat/assets/detector_models/{detector_id}/prediction” folder.

Child links: TRA-014

Attribute Value
platform Ubuntu 22.04.4 LTS
execution_type M
verification_method T
release alpha
complexity 2
test_data see referenced files