1 TCER: modelling TRA-001

This test case execution result (TCER) reports the outcome of verifying modelling artifacts.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify linked requirements

  • 0 = flawlesss: The data model of SATRAP-DL uses a data modelling language based on type theory, namely TypeQL.
  • 0 = flawlesss: SATRAP-DL relies on a database paradigm that allows for knowledge representation based on semantics and PERA model implemented by TypeDB.
  • 0 = flawlesss: SATRAP-DL supports querying the CTI SKB based on semantic criteria.
  • 0 = flawlesss: The data model of the CTI SKB is extensible and allows for the integration of new information.
  • 0 = flawlesss: The data model of the CTI SKB SHALL relies on a type-theoretic polymorphic entity-relation-attribute (PERA) data model to allow for the addition of new entities and relationships without requiring a schema migration.

Test case step 2: Check for alignment between system concept and implemented system

  • 0 = flawlesss: alignment confirmed upon reviewing design artifacts and comparing these against the implementation.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

  • N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-001 TC: Verify data modelling artifacts

Attribute Value
test-date 2025-03-25
tester AAT
defect-category 0 = flawless
passed-steps 2
release-version 0.1
verification-method R

2 TCER: SW engineering TRA-002

This test case execution result (TCER) reports the outcome of the verification of naming convention usage and adherence to the SOLID software engineering principles.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify consistent naming convention use

  • 1 = insignificant defect: based on a sample of the source files, most functions, classes and variables follow the PEP-8 naming convention consistently. Nevertheless, we did identify one problematic instance in the log_utils.py module, see the comments section below for more details.

Test case step 2: Verify adherence to SOLID

  • 0 = flawlesss: the 5 SOLID design principles are largely respected by the architectural modules.

Defect summary description

An insignificant defect was detected during test execution, i.e., thus assigning the overall highest defect category from the test step verdicts: 1 = insignificant defect

Please see the comments below for a few relevant observations.

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

The function definition testing(self, ...) in satrap/commons/log_utils.py: a function at the module level has a self parameter in its signature, as opposed to being used in instance methods within class definitions to refer to the instance of the class; see linked file for exact reference.

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect>
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

SOLID principles: Single responsibility principle (SRP), Open-closed principle (OCP), Liskov substitution principle (LSP), Interface integration principle (ISP), Dependency inversion principle (DIP).

satrap/commons/log_utils.py (line 69)

Parent links: TST-002 TC: Verify software engineering

Attribute Value
test-date 2025-03-25
tester AAT
defect-category 1 = insignificant defect
passed-steps 2
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method R

3 TCER: STIX and reasoning TRA-003

This test case execution result (TCER) reports the outcome of STIX and reasoning engine usage verification.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify DBMS technology

  • 0 = flawlesss: SATRAP-DL uses a DBMS technology that comes with a reasoning engine as a key integral part, namely TypeDB.

Test case step 2: Verify use of STIX 2.1

  • 0 = flawlesss: SATRAP-DL uses STIX 2.1 as the default standard format for CTI representation.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-003 TC: Verify STIX and reasoning engine

Attribute Value
test-date 2025-03-25
tester AAT
defect-category 0 = flawless
passed-steps 2
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method R,I

4 TCER: data model TRA-004

We analyze the SATRAP data model to verify adherence to that of STIX 2.1.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify data model alignment with STIX 2.1

  • 1 = insignificant defect: support for ingesting STIX 2.1 is implemented, providing a direct mapping of the imported data to equivalent concepts in the TypeDB database; however, custom and metadata objects are currently missing.

Defect summary description

Assigned defect category: 1 = insignificant defect

STIX 2.1 is currently not complete (custom properties and meta objects currently not handled), but sufficient coverage is provided for the alpha release.

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-004 TC: Verify data model

Attribute Value
test-date 2025-03-25
tester AAT
defect-category 1 = insignificant defect
passed-steps 1
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method A

5 TCER: centralized management TRA-005

We report on our inspection to verify centralized management of system parameters customization via a dedicated configuration file, and of log storage, exception types and error messages.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

SATRAP-DL enables customization of system parameters via a YAML file located at satrap/assets/satrap_params.yml, capturing logging mode, TypeDB database parameters (host, port, db name) and ETL default source files/paths.

Test case step 1: Verify centralized system parameterization

  • 0 = flawlesss: The user-controlled YAML file satrap_params.yml captures logging mode.

Test case step 2: Verify centralized parameterization for database connections

  • 0 = flawlesss: The user-controlled YAML file satrap_params.yml captures TypeDB database parameters (host, port, db name)

Test case step 3: Verify centralized parameterization file for managing file paths

  • 0 = flawlesss: The settings.py encapsulates paths to various resources used throughout the code.

Test case step 4: Verify designated logs storage location

  • 0 = flawlesss: SATRAP-DL stores its logs under satrap/assets/logs, with log files organized under subfolders named by date, which in turn contain timestamped logs files capturing the name of the log producing module.

Test case step 5: Verify centralized exception definitions

  • 0 = flawlesss: Exceptions are defined in a centralized manner and stored in satrap/commons/exceptions.py.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

satrap/assets/satrap_params.yml (line 3) satrap/settings.py satrap/commons/exceptions.py

Parent links: TST-005 TC: Verify centralized management

Attribute Value
test-date 2025-03-26
tester AAT
defect-category 0 = flawless
passed-steps 5
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method I

6 TCER: code clarity TRA-006

We report on our code inspections to validate the logging feature of the ETL subsystem.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify source code clarity according to the C5-DEC method and guidelines

  • 0 = flawlesss: The majority of the SATRAP-DL code base exhibits a high degree of consistency in terms of understandability, readability and being well-documented.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-006 TC: Verify code clarity

Attribute Value
test-date 2025-03-26
tester AAT
defect-category 0 = flawless
passed-steps 1
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method I

7 TCER: secure programming TRA-007

This test case execution report addresses the validation test case dealing with secure programming aspects and practices.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify components input validation

  • 2 = minor defect: while most SATRAP components receiving input perform some form of validation, we have identified a few discrepancies such as those in the ETL extractor module. The fetch functions in the Downloader and STIXExtractor do not validate the source URLs or file paths against a reference pattern or inclusion in any black or white lists. They do however perform syntactic checks directly and indirectly via the validate_file function in the file_utils module.

Test case step 2: Verify sanitization of input and output of data passing across trust boundaries

  • 0 = flawlesss: since integration with TIPs or other external sources residing outside the SATRAP trust boundary is currently not implemented as part of the alpha release (requirements for beta release), this sanitization requirements is considered to be satisfied.

Test case step 3: Verify resource liberation

  • 0 = flawlesss: the ETL subsystem and various functions making I/O operations correctly release resources, e.g., database network connections and file streams handled via with context managers.

Test case step 4: Verify SBOM usage

  • 0 = flawlesss: all software dependencies of SATRAP, thanks to use its of Poetry, are listed precisely in an inventory providing a software bill of material (SBOM), that lists all used libraries, their respective versions, along with the corresponding hashes (automatically generated lock file).

Test case step 5: Verify log string sanitization

  • 2 = minor defect: log strings are currently not sanitized and validated before being logging to prevent log injection attacks.

Log injection vulnerabilities can emerge when writing invalidated user input to log files can allow an attacker to forge log entries or inject malicious content into the logs; the data can enter an application from an untrusted source (N/A in the alpha release of SATRAP) or it can be written to an application or system log file. (applicable to the alpha release)

A note on log forging (source: OWASP)

In the most benign case, an attacker may be able to insert false entries into the log file by providing the application with input that includes appropriate characters. If the log file is processed automatically, the attacker can render the file unusable by corrupting the format of the file or injecting unexpected characters. A more subtle attack might involve skewing the log file statistics. Forged or otherwise, corrupted log files can be used to cover an attacker’s tracks or even to implicate another party in the commission of a malicious act.

Test case step 6: Verify secret storage

  • 0 = flawlesss: Manual and automated scans (SAST) confirm the absence of logged or hardcoded sensitive information in the source code such as passwords or entity identifiers.

Test case step 7: Verify data semantic integrity enforcement

  • 0 = flawlesss: the TypeDB engine, together with the SATRAP data model and TypeQL types (cti-skb-types.tql) enforce semantic integrity ensuring that relationships and constraints adhere to the intended meaning. These enable benefitting from measures such as data validation with respect to schemas and relationship constraints, technical possibility of automated checks for data redundancy and inference powered by a reasoning engine.

Defect summary description

Various minor issues have been identified, thus assigning the overall highest defect category from the test step verdicts: 2 = minor defect

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

satrap/etl/extract/extractor.py (line 51) satrap/etl/extract/extractor.py (line 88) satrap/commons/file_utils.py (line 33) satrap/commons/file_utils.py (line 68) satrap/assets/schema/cti-skb-types.tql

Parent links: TST-007 TC: Verify secure programming

Attribute Value
test-date 2025-03-26
tester AAT
defect-category 2 = minor defect
passed-steps 5
failed-steps 2
not-executed-steps 0
release-version 0.1
verification-method I,R

8 TCER: MITRE ATT&CK ingestion TRA-008

This test case execution report covers the validation test case specification on the ingestion of the MITRE ATT&CK data set.

Relevant test environment and configuration details

  • Software deviations: aligned with test case specification
  • Hardware deviations: aligned with test case specification

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify the execution of the SATRAP setup CLI command

  • 0 = flawlesss: the obtained result aligns with the expected outcome described in the linked validation test case specification.

Test case step 2: Verify the execution of the SATRAP etl CLI command

  • 0 = flawlesss: the obtained result aligns with the expected outcome described in the linked validation test case specification.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-008 Test setup + MITRE ATT&CK ingestion

Attribute Value
test-date 2025-03-27
tester AAT
defect-category 0 = flawless
passed-steps 2
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method T

9 TCER: ETL architecture TRA-009

This test case execution report covers the ETL subsystem; see the linked files for precise references to the cited code modules and classes mentioned below.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify data integration module for orchestration

  • 0 = flawlesss: As part of its dedicated ETL subsystem, SATRAP provides a module in charge of orchestrating (etlorchestrator.py), another for the ingestion (extractor.py) of datasets in STIX 2.1 and, finally, another module for the insertion (loader.py) of content into the SATRAP CTI SKB, powered by TypeDB.

Test case step 2: Verify the integration module for transforming data into STIX 2.1

  • 0 = flawlesss: As part of its dedicated ETL subsystem, SATRAP provides a module in charge of transforming (transformer.py) the ingested STIX data into the representation language of the CTI SKB schema, namely TypeQL.

Test case step 3: Verify data integration module for database operations and connections

  • 0 = flawlesss: SATRAP provides a dedicated data management package containing various related modules, with one in particular (typedbmanager.py) in charge of managing database operations and connections.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

satrap/etl/etlorchestrator.py (line 11) satrap/etl/extract/extractor.py satrap/etl/transform/transformer.py satrap/etl/load/loader.py satrap/datamanagement/typedb/typedbmanager.py

Parent links: TST-009 Verify ETL architecture

Attribute Value
test-date 2025-03-27
tester AAT
defect-category 0 = flawless
passed-steps 3
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method R

10 TCER: CTI SKB inference TRA-010

This test case execution report covers the SATRAP automated reasoning and inference capabilities; see the linked files for precise references to the cited files.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify that SATRAP-DL implements inference rules for automated reasoning

  • 0 = flawlesss: The SATRAP implementation provides a dedicated rules file (cti-skb-rules.tql). By analyzing the CTI SKB inference rule definition file and CTI SKB type definitions files stored in the satrap/assets/schema, we confirm the presence of the required artifacts enabling derivation of knowledge over existing relations in the CTI SKB powered by TypeDB.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

satrap/assets/schema/cti-skb-rules.tql

Parent links: TST-010 Verify CTI SKB inference rules

Attribute Value
test-date 2025-03-27
tester AAT
defect-category 0 = flawless
passed-steps 1
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method A

11 TCER: Jupyter Notebook frontend TRA-011

Relevant test environment and configuration details

  • Software deviations: aligned with test case specification
  • Hardware deviations: aligned with test case specification

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Open the source code of SATRAP-DL in Microsoft VS Code Studio using the Dev Containers extension

  • 0 = flawlesss: The project loaded successfully, with a bash session being the entry point selected by default, giving access to the SATRAP GNU/Linux container.

Test case step 2: Navigate to the VS Code Studio terminal and activate Python environment

  • 0 = flawlesss: The SATRAP container was successfully accessed, and the commands poetry shell and poetry install also successfully run.

Test case step 3: Navigate to the folder satrap/frontend/quick_start.ipynb and activate kernel

  • 0 = flawlesss: Jupyter Notebook located successfully and kernel activated.

Test case step 4: Run each cell from top to bottom in order

  • 0 = flawlesss: All cells executed, and their outputs were successfully compared to the expected output reference file quick_start-test-reference.ipynb.

Part 1 on "Starting with simple functions":

  • 0 = flawlesss: satrap = CTIanalysisToolbox(TYPEDB_SERVER_ADDRESS, DB_NAME)
  • 0 = flawlesss: print(satrap.get_sdo_stats())
  • 0 = flawlesss: print(satrap.mitre_attack_techniques())
  • 0 = flawlesss: print(satrap.mitre_attack_mitigations())
  • 0 = flawlesss: cell "Get information on a specific MITRE ATT&CK element (technique, group, software, etc.) using its MITRE ATT&CK id."
  • 0 = flawlesss: cell "Get information about a STIX object using its STIX id."
  • 0 = flawlesss: cell "Retrieve mitigations explicitly associated to a specific technique using its STIX id."

Part 2 on "CTI analysis through automated reasoning":

  • 0 = flawlesss: cell "Get statistics on the usage of ATT&CK techniques by groups. The output of this function is the same as when running the command satrap techniques on the CLI."
  • 0 = flawlesss: cell display(satrap.techniques_usage(infer=True))
  • 0 = flawlesss: cell techniques = satrap.techniques_used_by_group("G0025", infer=True)
  • 0 = flawlesss: cell display(satrap.related_mitigations(group_name="BlackTech"))

Subsection "Explanation of inferred knowledge"

  • 0 = flawlesss: 1st cell starting with rel_explanation = satrap.explain_if_related_mitigation("G0098", "course-of-action--20a2baeb-98c2-4901-bad7-dc62d0a03dea").
  • 0 = flawlesss: 2nd cell starting with reason = satrap.explain_related_techniques("ZIRCONIUM", "T1059.006").
  • 0 = flawlesss: last cell starting with dg_explanation = satrap.explain_techniques_used_by_group("G0071", "Domain Groups").

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

The function signature t1 = satrap.get_attck_concept_info("T1027.001") contains a typo: attck -> attack

Functions are named inconsistently: there is a mix of verb-based (preferred as per C5-DEC conventions) and noun-based naming, e.g.,

  • satrap.get_sdo_stats()
  • satrap.mitre_attack_techniques()
  • satrap.mitigations_for_technique()
  • satrap.get_attck_concept_info()
  • satrap.search_stix_object()

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

docs/specs/TST/assets/quick_start-test-reference.ipynb

Parent links: TST-011 Test Jupyter notebook frontend

Attribute Value
test-date 2025-03-27
tester AAT
defect-category 0 = flawless
passed-steps 4
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method T

12 TCER: ETL logging TRA-012

We ran tests to validate the logging feature of the ETL subsystem.

Relevant test environment and configuration details

  • Software deviations: aligned with test case specification
  • Hardware deviations: aligned with test case specification

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify ETL logging by running ./satrap.sh etl to ingest the latest version of the MITRE ATT&CK Enterprise data set

0 = flawlesss: The generated log, i.e., test-evidence-log.txt, stored in the satrap/assets/logs folder was checked (stored under date folders and timestamped files according to ETL execution time) and the following items were validated:

  • a log entry is generated at the beginning of the job, indicating the start time
  • each log entry recording an event comes with a log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
  • there is at least a log entry for each ETL phase (i.e., extraction, transformation and loading) describing the status in terms of success/failure and some minimal hint or additional information explaining or complementing the execution status.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

docs/specs/TRA/assets/test-evidence-log.txt

Parent links: TST-012 Test ETL logging

Attribute Value
test-date 2025-03-29
tester AAT
defect-category 0 = flawless
passed-steps 1
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method T

13 TCER: CM settings TRA-013

We report on our inspection of the centralized settings file to verify that some of its content is read from a user-controller configuration management file allowing configuration management without requiring software rebuild.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Navigate to the settings.py

  • 0 = flawlesss: The file settings.py was located at satrap/settings.py.

Test case step 2: Navigate to the YAML configuration management

  • 0 = flawlesss: The configuration management file satrap_params.yml exposed to the user was found at satrap/assets/satrap_params.yml.

Test case step 3: Check that the YAML file is read into memory (e.g., in a Python dictionary).

  • 0 = flawlesss: this was validated the settings.py file read_yaml(SATRAP_PARAMS_FILE_PATH); see linked files section for the precise, automatically retrieved line number.

Test case step 4: Check that at least one settings parameter read into memory

  • 0 = flawlesss: parameters in settings.py are populated from the in-memory copy of the satrap_params.yml file, with an example given below
HOST = satrap_params_dict.get('typedb').get('host','typedb')

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

satrap/settings.py (line 34)

Parent links: TST-013 Inspect settings for CM

Attribute Value
test-date 2025-03-28
tester AAT
defect-category 0 = flawless
passed-steps 4
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method I

14 TCER: SATRAP CLI TRA-014

We report on our tests carried out using the SATRAP command line interface (CLI) to verify that it provides at least the commands specified in the software requirement specification (SRS) that the linked test case specification traces to.

Relevant test environment and configuration details

  • Software deviations: aligned with test case specification
  • Hardware deviations: aligned with test case specification

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Run ./satrap.sh rules

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Test case step 2: Run ./satrap.sh stats

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Test case step 3: Run ./satrap.sh techniques

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Test case step 4: Run ./satrap.sh mitigations

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Test case step 5: Run ./satrap.sh search campaign--0c259854-4044-4f6c-ac49-118d484b3e3b

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Test case step 6: Run ./satrap.sh info_mitre T1027.001

  • 0 = flawlesss: obtained result consistent with the expected outcome specified in the linked test case.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-014 Test command line interface (CLI)

Attribute Value
test-date 2025-03-28
tester AAT
defect-category 0 = flawless
passed-steps 6
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method T

15 TCER: Open-source TIP adoption TRA-015

We reviewed the SATRAP design artifacts to validate its adoption of open-source TIPs by design, specifically MISP and OpenCTI.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify SATRAP adoption of open-source TIPs

  • 0 = flawlesss: we confirmed upon reviewing the SATRAP system concept documents that it adopts both MISP and OpenCTI as TIPs.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-017 Verify open-source TIP integration

Attribute Value
test-date 2025-03-28
tester AAT
defect-category 0 = flawless
passed-steps 1
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method R

16 TCER: release + licensing TRA-016

We report on our inspection of SATRAP release and licensing model.

Relevant test environment and configuration details

  • Software deviations: N/A
  • Hardware deviations: N/A

Test execution results

Here we report the results in terms of step-wise alignments or deviations with respect to the expected outcome of the covered test case.

Test case step 1: Verify public open-source release of SATRAP source code

  • 0 = flawlesss: The entire source code of SATRAP was confirmed to be released on GitHub via a public repository, as per project agreements.

Test case step 2: Verify licenses of 3rd-party libraries

  • 0 = flawlesss: SATRAP software library dependencies do not restrict the privileges granted by the license selected for SATRAP-DL.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawlesss

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Comments

N/A

Guide

  • Defect category: 0 = flawlesss; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
  • Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TST-018 Verify release and licensing

Attribute Value
test-date 2025-03-28
tester AAT
defect-category 0 = flawless
passed-steps 2
failed-steps 0
not-executed-steps 0
release-version 0.1
verification-method I