1 System structure overview ARC-001

This UML composite structure diagram shows the overview of SATRAP-DL along with the systems on which it has some dependency.

Composite structure diagram of SATRAP-DL

Roughly, the components are described as follows:

Component name Component description
CTI Knowledge representation system Semantic Knowledge Base of CTI (CTI SKB) defined on a strongly-typed data model, plus an automated logic-based reasoning engine.
Data manager Manages the interactions and connection to the knowledge base
ETL module Enables ingesting data from diverse categories into the CTI SKB, for instance, cybersecurity knowledge (e.g., datasets from MITRE ATT&CK), behavioral data (from SIEMs, SOARs, etc.) and external CTI (from platforms like MISP).
CTI analysis engine Implements queries tailored for the automation of Cyber Threat Intelligence (CTI) analysis tasks.
Controller Responsible for handling the interaction between the SATRAP-DL management console and the ETL module .
SATRAP-DL services Make SATRAP-DL's functionality accessible via a Python library and a language-independent API.
SATRAP-DL frontend A suite of user interfaces for executing and visualizing results of analytic queries over the CTI SKB and performing data management and admin tasks in the CTI SKB.

More details are available as part of the system concept in the file 2B1D_REP_CyFORT-SATRAP-DL-SystemArchitecture_v1.1.

Parent links: SRS-001 Data modelling language, SRS-002 Database paradigm, SRS-008 ETL subsystem, SRS-010 Database manager, SRS-011 Ingestion of organizational CTI, SRS-012 Inference rules, SRS-014 Native reasoning engine, SRS-015 Jupyter Notebook frontend, SRS-017 Integration of behavioral data, SRS-045 CTI analysis engine, SRS-046 CTI analysis toolbox

Child links: SWD-001 Top-level ETL design

2 Logical view of SATRAP ARC-002

The following UML package diagram depicts the logical view of SATRAP (Semi-Automated Threat Reconnaissance and Analysis Platform), the system to be built in SATRAP-DL.

Package diagram of SATRAP-DL

Details on the architecture are available as part of the system concept in the file 2B1D_REP_CyFORT-SATRAP-DL-SystemArchitecture_v1.1.

Parent links: SRS-001 Data modelling language, SRS-002 Database paradigm, SRS-008 ETL subsystem, SRS-010 Database manager, SRS-011 Ingestion of organizational CTI, SRS-014 Native reasoning engine, SRS-015 Jupyter Notebook frontend, SRS-017 Integration of behavioral data, SRS-045 CTI analysis engine, SRS-046 CTI analysis toolbox

Child links: SWD-001 Top-level ETL design

3 ETL high-level design ARC-003

An overview of the main external systems and internal components of SATRAP involved in the ETL process is shown in the following diagram.

ETL system components

Parent links: SRS-006 Integration of common CTI, SRS-008 ETL subsystem, SRS-009 ETL Transformer, SRS-010 Database manager, SRS-011 Ingestion of organizational CTI, SRS-013 STIX 2.1 data model, SRS-020 System configuration file, SRS-023 CTI representation in STIX 2.1, SRS-024 Design and implementation principles, SRS-028 Input validation, SRS-029 Input sanitization

Child links: SWD-001 Top-level ETL design, SWD-002 STIX-specific ETL design, SWD-003 ETL system flow, SWD-004 TypeDB utilities, SWD-005 Transformer class diagram, SWD-006 Transformer flow, SWD-007 ETL full class diagram

4 ETL components ARC-004

The following diagram depicts the main components of the ETL system.

ETL system components

Roughly, the ETLOrchestrator is in charge of the logic for executing the ETL process assisted by an Extractor, a Transformer and a Loader.

A suitable Extractor fetches data from an external source and creates and stores a datasource in STIX 2.1 format in a predefined folder. For the initial version, we will only consider an extractor for datasources already in STIX 2.1, namely the STIXExtractor. In future phases, the integration of data in other formats can be supported by extending the architecture with new Extractors.

The ETL subsystem interacts with the following components:

  • ETL runner: this is the component that triggers the ETL process according to predefined settings.
  • STIX datasets: a predefined folder in the file system storing datasets in STIX2.1 JSON format.
  • CTI SKB: the database of SATRAP in TypeDB.
  • Data Management: includes components aimed at handling data, such as the DB manager which manages the connections and operations over the CTI SKB. Some of the functions in this class are: create_db, setup_schema, load_db_data, insert and delete.

Parent links: SRS-001 Data modelling language, SRS-002 Database paradigm, SRS-005 NoSQL data model, SRS-006 Integration of common CTI, SRS-008 ETL subsystem, SRS-009 ETL Transformer, SRS-010 Database manager, SRS-013 STIX 2.1 data model, SRS-020 System configuration file, SRS-023 CTI representation in STIX 2.1, SRS-024 Design and implementation principles, SRS-028 Input validation, SRS-029 Input sanitization

Child links: SWD-001 Top-level ETL design, SWD-002 STIX-specific ETL design, SWD-003 ETL system flow, SWD-004 TypeDB utilities, SWD-005 Transformer class diagram, SWD-006 Transformer flow, SWD-007 ETL full class diagram