1 Test accessing and browsing the CC Database TST-001

Setup

  1. boot C5-DEC CAD
  2. navigate to '3 - CCT: Common Criteria toolbox' > 'Browse Common Criteria'

Test Steps

  1. follow steps in SRS
  2. on a random basis check (for at least 3 elements) that the displayed data corresponds to the associated Common Criteria element in the official PDFs of version 3.1 R5: CC Part 2 or CC Part 3.

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-001 Access and Browse Database for CC Security Components

Child links: TRA-001

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition c5dec installed
expected_outcome (i) No errors encountered while browsing the database. (ii) Randomly sampled selection data corresponds to description in the respective Common Criteria PDF.
verification_method Test
release alpha

2 Test query the CC Database TST-002

Setup

  1. boot C5-DEC CAD
  2. navigate to '3 - CCT: Common Criteria toolbox' > 'Common Criteria Browser'

Test Steps:

  1. follow the steps in SRS
  2. repeat steps multiple times for different queries (at least 5) and match displayed content with respective Common Criteria PDF (version 3R5).

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-002 Query the Database of Security Components.

Child links: TRA-002

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition C5Dec installed
expected_outcome None
verification_method Test
release alpha

3 Test exporting Security Components TST-003

Setup

  1. In the TUI, go to '3 - CCT: Common Criteria toolbox'
  2. Select 'Browse Common Criteria' or 'Create Evaluation Checklist'

Test Steps

  1. follow steps in SRS
  2. repeat multiple times and change selection and export settings.
  3. on a random basis check that the exported Security Requirements match respective Security Requirements in Common Criteria PDF (version 3R5).

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-003 Export Security Components

Child links: TRA-003

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition a working doorstop repository
expected_outcome File containing respective Security Requirements correctly exported according to specified settings.
verification_method Test
release alpha

4 Test tailoring Security Requirements TST-004

TBD

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-004 Tailor Security Requirements.

Child links: TRA-004

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
release alpha

5 Test navigating the Knowledge Base TST-005

Setup

  • None

Test steps

  1. follow steps in SRS
  2. repeat for multiple Knowledge Articles

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-014 Access and Navigation Through Knowledge Base

Child links: TRA-005

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Review
release alpha

6 Test comprehensiveness of Knowledge Base TST-006

Setup

  • None

Test steps

  1. follow steps in SRS
  2. repeat for multiple Knowledge Articles

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-016 Interconnected Framework and Seamless Navigation

Child links: TRA-006

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Review
release alpha

7 Test interconnection of Knowledge Base TST-007

Setup

  1. None

Test steps

  1. follow steps in SRS
  2. repeat for multiple Knowledge Articles

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-016 Interconnected Framework and Seamless Navigation

Child links: TRA-007

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Review
release alpha

8 Test currency of Knowledge Base TST-008

Setup

  1. None

Test steps

  1. follow steps in SRS
  2. repeat for multiple Knowledge Articles

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-017 Availability of Pragmatic Guidance and Best Practices

Child links: TRA-008

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Review
release alpha

9 Test cosmetic features of Knowledge Base TST-009

Setup

  1. None

Test steps

  1. follow steps in SRS'
  2. repeat for multiple Knowledge Articles

c5dec/assets/database/KnowledgeBase/0_MapofContent.md

Parent links: SRS-019 Integration of Multimedia Elements in Knowledge Articles, SRS-020 Integrating Links to FAQ Section, SRS-021 Provision of General Link to a CC-Specific Forum

Child links: TRA-009

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
release alpha

10 Inspect CC Database-DTD mapping TST-010

Setup

  1. None

Test steps

  1. Inspect source code 'cct.py' and verify that each '!ELEMENT' in the CC DTD is correctly implemented, i.e, verify that:
  • each '!ELEMENT' is implemented as a class object,
  • '#REQUIRED' attributes are implemented as object attributes and populated with the object's _build_attributes() method.
  • child elements of each '!ELEMENT' are implemented as object attributes and populated with the object's _build_children() method.
  • each object's is_valid() method validates that the required attributes (labelled as '#REQUIRED' in the DTD) are (correctly) populated.
  • semantic relationships between CC concepts are preserved in the implementation, i.e., hierarchical and dependency relationships between CC concepts are appropriately implemented, e.g., as parent-child relations.

c5dec/core/cct.py (line 630) c5dec/assets/database/SecurityControls/cc3R5.xml

Parent links: SRS-022 Data Storage and Format Mapping

Child links: TRA-010

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Inspection
release alpha

11 Test correctness of CC Database TST-011

Setup

  1. Use the CLI to export CC objects using

    $ poetry run c5dec view <ID> #to print to console

    $ poetry run c5dec view <ID> > <filepath>/<filename>.md # to export to markdown

  2. Use the TUI to view CC objects

    • navigate to '3 - CCT: Common Criteria Toolbox' > 'Common Criteria Browser'

Test steps

  1. Export or view mulitple CC objects either via CLI or TUI and check that content and relationships correspond to the Common Criteria PDF.

Parent links: SRS-022 Data Storage and Format Mapping

Child links: TRA-011

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome Exported or viewed data correctly reflects content and relationships as defined in the CC PDF.
verification_method Test
release alpha

12 Test validity and conistency of bidirectional transformation TST-012

Setup:

  1. None

Test steps:

  1. Export CC Document stored in the internal data representation to XML (cc_export1.xml)
  2. Validate the exported XML against the CC DTD.
  3. Load the exported and validated XML into the internal data representation.
  4. Again export the loaded CC Document to XML (cc_export2.xml) and validate against the CC DTD.
  5. Repeat if desired.
  6. Verify that all exported XMLs are identical.

Note: Exported XML files will differ from the official CC XML since irrelevant information (e.g., patchinfos legalnotice, pagebreaks, biblioentries, glossentries, etc.) are ignored.

c5dec/assets/database/SecurityControls/cc3R5.dtd c5dec/assets/database/SecurityControls/cc3R5.xml

Parent links: SRS-022 Data Storage and Format Mapping, SRS-023 Bidirectional Transformation and Consistency

Child links: TRA-012

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Validate any/all CC XML independently against the DTD file (./c5dec/assets/database/SecurityControls/cc3.xml)
expected_outcome All exported CC XML files are valid and identical.
verification_method Test
release alpha

13 Test threats, risks, and countermeasures database TST-013

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-024 Threats, Risks, and Countermeasures Database

Child links: TRA-013

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

14 Test EUCC support TST-014

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-025 Support Mechanism for EUCC Additional Evaluation Evidence

Child links: TRA-014

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

15 Test availabilty of CSA Article 51 defined Security Objectives TST-015

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-026 Storing Generic Security Objectives of CSA Article 51

Child links: TRA-015

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

16 Test uniformity of storage mechanisms in CCT module TST-016

Setup

  1. None

Test steps

  1. follow steps in SRS
  2. repeat for multiple items.

Parent links: SRS-027 Unified Storage Mechanism for CC Artifacts and Security Elements

Child links: TRA-016

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome All internally stored CC artefacts are stored in C5-DEC document-based data storage format and can be managed centrally.
verification_method Test
release alpha

17 Test automated rationale and traceability matrix generation TST-017

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-028 Automated Rationale and Traceability Matrix Generation

Child links: TRA-017

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

18 Test verificaiton of rationales and traceability matrices TST-018

Setup

  1. None

Test steps

  1. follow steps in SRS

Parent links: SRS-029 Verification of Rationales and Traceability Matrices

Child links: TRA-018

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

19 Test automated consistency and completeness checks TST-019

Setup

  1. None

Test steps

  1. follow steps in SRS

Parent links: SRS-030 Detailed Consistency and Completeness Checks

Child links: TRA-019

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

20 Test automated validation TST-020

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-031 Automated Validation of Relationships, Attributes, and Dependencies

Child links: TRA-020

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

21 Test aggregation of SARs and Work Units TST-021

Setup

1.a Using the TUI navigate to '3 - CCT: Common Criteria toolbox' > 'Evaluation Checklist' and load an Evaluation Checklist.

1.b Using the CLI run the following command to list all Evaluation items.

`$ poetry run c5dec checklist <prefix> -l`

To select an Item run:
`$ poetry run c5dec checklist <prefix> --edit <item-id> [--editor code]`

Test steps

  1. Follow steps in SRS.
  2. Repeat for multiple SARs.

Parent links: SRS-032 Seamless Aggregation and Presentation of SARs and Work Units

Child links: TRA-021

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Evaluation checklist created and stored.
expected_outcome None
verification_method Test
release alpha

22 Test API provision for threat import TST-022

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-033 API Provision for Threat Importation

Child links: TRA-022

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

23 Test transforming imported threats to CC-conformant format TST-023

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-034 Transformation of Imported Threats to CC-conformant Format

Child links: TRA-023

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

24 Test automated creation of Evaluation Checklist TST-024

Setup

  1. when using the TUI navigate to '3 - CCT: Common Criteria Toolbox' > 'Create Evaluation Checklist'

  2. when using the CLI run the following command

    foo@bar$ poetry run c5dec checklist -c <prefix> --id <Component/Package Ids>

Valid set: ACO_COR.1, ACO_DEV.1, ACO_REL.1, ALC_CMC.1, ALC_CMS.1

Invalid Set: ACO_COR.1, ACO_DEV.1, ALC_CMC.1, ALC_CMS.1

Augmented Set: EAL 1, ACO_DEV.1, ACO_REL.1

Test steps

  1. follow steps in SRS for the selections above.
  2. OPTIONAL: follow steps in SRS for a randomly selected set of components.

Parent links: SRS-035 Automated Evaluation Checklist Creation

Child links: TRA-024

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome (i) Successfully created Evaluation Checklist for the valid set (ii) Invalid set successfully detected to be invalid. (iii) Successfully created Evaluation Checklist for the augmented set (iv) Evaluation Evidence Documentation Index created for valid sets and stored under ./evaluations/<prefix>
verification_method Test
release alpha

25 Test evaluation progress tracking TST-025

Setup

  1. when using the TUI navigate to '3 - CCT: Common Criteria Toolbox' > 'Evaluation Checklist'

  2. when using the CLI run the following commands

    $ poetry run c5dec checklist <prefix> -s # returns current status

    $ poetry run c5dec checklist <prefix> -l # returns list of Evaluation items.

    $ poetry run c5dec checklist <prefix> --edit <item ID> # edit Evaluation Item

Test steps

  1. follow steps in SRS.
  2. repeat for multiple Evaluation Items.

Parent links: SRS-036 Evaluation Progress Tracking

Child links: TRA-025

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

26 Test Work Unit-Artifact linking TST-026

Setup

  1. when using the TUI navigate to '3 - CCT: Common Criteria Toolbox' > 'Evaluation Checklist'

  2. when using the CLI run the following commands

    $ poetry run c5dec checklist <prefix> -s # returns current status

    $ poetry run c5dec checklist <prefix> -l # returns list of Evaluation items.

    $ poetry run c5dec checklist <prefix> --edit <item ID> # edit Evaluation Item

    $ poetry run c5dec view <item ID>

Test steps

  1. follow steps in SRS.

Parent links: SRS-037 Work Unit Artifact Linking

Child links: TRA-026

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

27 Test automated generation of Observation Reports TST-027

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-038 Automated OR Generation

Child links: TRA-027

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

28 Test automated generation of Evaluation Technical Report TST-028

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-039 Automated ETR Generation

Child links: TRA-028

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

29 Test flagging failed Work Units and affected artifacts TST-029

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-040 Flagging and Addressing Failed Work Units with Cascading Flags

Child links: TRA-029

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

30 Test auditability of Evaluation Items TST-030

Setup

  • when using the TUI navigate to '3 - CCT: Common Criteria Toolbox' > 'Evaluation Checklist'

    1. Load Evaluation Checklist
    2. randomly select Evaluation Items and edit these to set a verdict.

  • when using the CLI

    1. run the following command for several Evaluation Items and edit their verdicts

      $ poetry run c5dec checklist <prefix> --edit <item ID> [--editor code]

    2. manually update the Evaluation Checklist with $ poetry run c5dec checklist <prefix> --update

Test steps

  1. Verify that index.json ./evaluations/<prefix>/index.json correctly reflects the changes, i.e., hash value and verdict matches the 'reviewed' and 'verdict' value of the corresponding item.
  2. Verify that changes are correctly reflected in corresponding Doorstop items.
  3. Logging inherently covered by git, e.g., by using $ git log --follow -p -- ./path/to/file.ext

Parent links: SRS-041 Logging evaluated Work Units

Child links: TRA-030

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
precondition Run TST-024 prior to have access to an Evaluation Checklist.
expected_outcome None
verification_method Test
release alpha

31 Test extended data model TST-031

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-042 Extend Data Model Across Entire CC Ecosystem

Child links: TRA-031

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

32 Test CC templates TST-032

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-043 Provide CC Document Templates

Child links: TRA-032

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

33 Test validation of hierarchies and dependencies of Security Component sets TST-033

Setup

  1. In the TUI navigate to '3 - CCT: Common Criteria Toolbox' > 'Create Evaluation Checklist'

Test steps

  1. Select items from the following component sets and validate that the expectations in the SRS are met:
    • valid: ACO_COR.1 ACO_DEV.1 ACO_REL.1 ALC_CMS.1 ALC_CMC.1
    • valid (EAL2): ADV_ARC.1 ADV_FSP.2 ADV_TDS.1 AGD_OPE.1 AGD_PRE.1 ALC_CMC.2 ALC_DEL.1 ASE_INT.1 ASE_CCL.1 ASE_SPD.1 ASE_OBJ.2 ASE_ECD.1 ASE_REQ.2 ASE_TSS.1 ATE_COV.1 ATE_FUN.1 ATE_IND.2 AVA_VAN.2
    • invalid: ADV_ARC.1 ADV_FSP.2 ADV_TDS.1 AGD_OPE.1 AVA_VAN.2 corrected: ADV_ARC.1 ADV_FSP.2 ADV_TDS.1 AGD_OPE.1 AGD_PRE.1 AVA_VAN.2
    • invalid: ACO_COR.1 ACO_DEV.1 ACO_REL.1 corrected: see first set

Parent links: SRS-044 Validate Hierarchies and Dependencies in Security Components

Child links: TRA-033

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome - Valid selections are detected as valid selections.\n - invalid selection are detected to be invalid and the potentially valid set matches the\ respective corrected set
verification_method Test
release alpha

34 Test generation of Impact Analysis Report TST-034

Setup

  1. None

Test steps

  1. follow steps in SRS.

Parent links: SRS-045 Generate Impact Analysis Reports for Certification Maintenance.

Child links: TRA-034

Attribute Value
platform WSL/GNU/Linux Ubuntu 20/Linux Ubuntu 22.04
expected_outcome None
verification_method Test
release alpha

35 Run the GUI from the CLI with option --gui TST-035

Test steps

  1. In the CLI run c5dec --gui

Child links: TRB-001

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition c5dec has been successfully installed
expected_outcome The GUI of the CCT module opens in the default browser
verification_method Test
release beta

36 Run the GUI from the CLI with option -g TST-036

Test steps

  1. In the CLI, press ctrl+c to quit the command run in TST-035.
  2. Run c5dec -g

Child links: TRB-002

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition Run this test immediatly after TST-035
expected_outcome The GUI of the CCT module opens in the default browser
verification_method Test
release beta

37 The user manual contains guides for the functionality added in the Beta release TST-037

Test steps

  1. Open the user manual
  2. Verify that the information in the "Quick start" section corresponds to the Beta release.
  3. Validate the information corresponding to the Beta release in the "CCT" section.
  4. Validate the information corresponding to the Beta release in the "SSDLC" section.
  5. Validate the information corresponding to the Beta release in the "Transformer" section.

Child links: TRB-003, TRB-031

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome The information in the user manual is correct and reflects the updates made for the Beta release.
verification_method Review
release beta

38 Browse the CC database in the GUI TST-038

Test steps

  1. In the home page of the GUI, select "SAR" in the dropdown list "Requirement type".

    • Expected outcome: the "Class" dropdown list gets populated with the CC assurance classes (ASE,ALC,ADV,AGD,ATE,AVA,ACO,APE,ACE)

  2. Select "ALC" in the dropdown list of "Class"

    • Expected outcome: all the families mentioned in the content text area are shown in the "Familiy" dropdown list

  3. Select "Family: ALC_LCD"

    • Expected outcome: The "Component" dropdown list gets populated with "ALC_LCD.1" and "ALC_LCD.2"

  4. Select "Component: ALC_LCD.2"

    • Expected outcome: The "Element" dropdown list gets populated with elements corresponding to the component "ALC_LCD.2"

Child links: TRB-004

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome Upon selection of an item, the subsequent dropdown list is updated automatically as described in the test steps.
verification_method Test
release beta

39 The CC browser displays the content of the selected items TST-039

Test steps

  1. In the home page of the GUI, click on the "Toggle preview" button.
  2. Select items from diverse classes, families, components and elements.

Child links: TRB-005

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome (i)The content displayed in the text area provides information about the selected items. (ii) The content in the toggle area corresponds to the text displayed in the text area.
verification_method Test
release beta

40 Copy content in markdown from the CC browser text area TST-040

Test steps

  1. Select any class, family, component and/or element from the dropdown lists.
  2. Click on the "Copy" button below the text area.
  3. Open a text editor.
  4. Paste the content (e.g., with ctrl+v)

Child links: TRB-006

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome The copied text is pasted into the text editor.
verification_method Test
release beta

41 Expand the text including information of all the descendents of the selected item TST-041

Test steps

  1. Select requirement type "SFR", class "FMT" and family "FMT_REV" in the dropdown lists.
  2. Click the toggle button to verify that the content corresponds to the description of the FAU Security audit family.
  3. Click on the button "Expand selection"
  4. Verify that the displayed content has been updated including information related to the family's components and elements, i.e., FMT_REV.1, FMT_REV.1.1 and FMT_REV.1.2.

Child links: TRB-007

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome All the descendants (families, components, elements) of a selected item are displayed
verification_method Test
release beta

42 Create spreadsheet evaluation checklist of an assurance class of CCv3.1R5 from the GUI TST-042

Test steps

  1. Go to the "Common Criteria Laboratory" tab.
  2. Select CC version: "3R5", Assurance class: "ALC" (selection of different classes is possible).
  3. Click on the "Create checklist" button.

Update 18-07-2024

The expected outcome is updated as follows:

(ii) A file with name format chklist-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec/export; this file captures all the work units from the ALC class.

(iii) A success message appears in the GUI informing of the assurance class and CC version exported, as well as the export output folder.

Child links: TRB-008, TRB-029

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome (i) A folder containing markdown files corresponding to the checklist items of ALC is created at $C5DEC_ROOT/evaluation. (ii) A file with name format chklist-<aaaammdd>-<hhmmss>-export-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec; this file captures all the work units from the ALC class. (iii) A success message appears in the GUI.
verification_method Test, Review
release beta

43 Create spreadsheet evaluation checklist of all classes of CCv3.1R5 from the GUI TST-043

Test steps

  1. Go to the "Common Criteria Laboratory" tab.
  2. Select CC version: "3R5", Assurance class: "ALL-CLASSES".
  3. Click on the "Create checklist" button.

Update 18-07-2024

The expected outcome is updated as follows:

(ii) A file with name format chklist-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec/export; this file captures all the work units in CCv3.1R5.

(iii) A success message appears in the GUI informing of the assurance class and CC version exported, as well as the export output folder.

Child links: TRB-009, TRB-030

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome (i) A folder containing markdown files corresponding to the checklist items of all the classes in CCv3.1R5 is created at $C5DEC_ROOT/evaluation. (ii) A file with name format chklist-<aaaammdd>-<hhmmss>-export-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec; this file captures all the work units in CCv3.1R5. (iii) A success message appears in the GUI.
verification_method Test, Review
release beta

44 Create spreadsheet evaluation checklist of an assurance class of CCv2022R1 from the GUI TST-044

Test steps

  1. Go to the "Common Criteria Laboratory" tab.
  2. Select CC version: "2022R1", Assurance class: "ALC" (test for different classes if desired).
  3. Click on the "Create checklist" button.

Child links: TRB-010

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome (i) A folder containing markdown files corresponding to the checklist items of ALC is created at $C5DEC_ROOT/evaluation. (ii) A file with name format chklist-<aaaammdd>-<hhmmss>-export-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec; this file captures all the work units from the ALC class. (iii) A success message appears in the GUI.
verification_method Test, Review
release beta

45 Create spreadsheet evaluation checklist of all classes of CC2022R1 from the GUI TST-045

Test steps

  1. Go to the "Common Criteria Laboratory" tab.
  2. Select CC version: "2022R1", Assurance class: "ALL-CLASSES".
  3. Click on the "Create checklist" button.

Child links: TRB-011

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The GUI has been launched from the terminal
expected_outcome (i) A folder containing markdown files corresponding to the checklist items of all the classes in CCv2022R1 is created at $C5DEC_ROOT/evaluation. (ii) A file with name format chklist-<aaaammdd>-<hhmmss>-export-<aaaammdd>-<hhmmss>.xlsx is created at $C5DEC_ROOT/c5dec; this file captures all the work units in CCv2022R1. (iii) A success message appears in the GUI.
verification_method Test, Review
release beta

46 Navigate using the links in the GUI TST-046

Test steps

  1. Click on the three buttons in the footer (CC Browser, CC Lab and C5DEC on GitHub).
  2. Go to the "Common Criteria Tab"
  3. Click on all the links in the CC Browser.
  4. Go to the "Common Critera Laboratory" tab.
  5. Click on all the links in the CC Lab.
  6. Go to the "Documentation" tab.
  7. Click on all the links in the tab.
  8. Go to the "About" tab.
  9. Click on all the links in tab.

Child links: TRB-012

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome All the links are valid and link to the expected websites or pages as per the description (text of the link).
verification_method Test
release beta

47 Export to spreadsheet an evaluation checklists with all the assurance classes of CCv3.1R5 from the CLI TST-047

Test steps

  1. Run c5dec export -h

    • Expected outcome: usage information for the export command is displayed

  2. Run c5dec export trp-047-ev 3R5

    • Expected outcome: a spreadsheet containing all the work units in CCv3.1R5 is created at $C5DEC_ROOT/c5dec/trp-047-ev-<yyyymmdd>-<hhmmss>

Child links: TRB-013, TRB-028

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition c5dec has been succesfully installed
expected_outcome All the expected outcomes described in the test steps are observed.
verification_method Test
release beta

48 Export to spreadsheet an evaluation checklists with all the assurance classes of CC2022R1 from the CLI TST-048

Test steps

  1. Run c5dec export checklist 2022R1

Child links: TRB-014, TRB-027

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition c5dec has been succesfully installed
expected_outcome A spreadsheet containing all the work units in CCv2022R1 is created at $C5DEC_ROOT/c5dec/checklist-<yyyymmdd>-<hhmmss>
verification_method Test
release beta

49 Export to spreadsheet an evaluation checklists with some assurance classes of CCv3.1R5 from the CLI TST-049

Test steps

  1. Run c5dec export chcklst3R5 3R5 -c ACE AVA

Child links: TRB-015

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome The evaluation checklist for classes ACE and AVA of CCv3.1R5 is created in a spreadsheet at '$C5DEC_ROOT/c5dec'
verification_method Test
release beta

50 Export to spreadsheet an evaluation checklists with some assurance classes of CC2022R1 from the CLI TST-050

Test steps

  1. Run c5dec export chcklst2022R1 2022R1 -c ACE AVA

Child links: TRB-016

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome The evaluation checklist for classes ACE and AVA of CC2022R1 is created in a spreadsheet at '$C5DEC_ROOT/c5dec'
verification_method Test
release beta

51 Export command: messages for invalid input TST-051

Test steps

Run the following commands and validate that the output is a message informing of the corresponding erroneous situation:

  1. c5dec export chcklst2022R1 other: the selected version is invalid
  2. c5dec export tst1 3R5 -p ACO_REL: the input argument must be an assurance component
  3. c5dec export tst1 3R5 -c ACO_REL.2: the input argument must be an assurance class
  4. c5dec export tst1 3R5 -p ACO_REL.7: invalid assurance component
  5. c5dec export tst1 3R5 -c ACO_RER: invalid assurance class

Note: the messages displayed do not have to be an exact match, but to inform about the described situation.

Child links: TRB-017, TRB-026

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome All the commands display an error message informing the user about the cause of the error
verification_method Test
release beta

52 Export to spreadsheet an evaluation checklists with some assurance components of CCv3.1R5 from the CLI TST-052

Test steps

Run the following commands and validate the corresponding described output:

  1. c5dec export chckComp3R5 3R5 -p ACO_REL.1: the evaluation checklist for component ACO_REL.1 of CCv3.1R5 is created in a spreadsheet at $C5DEC_ROOT/c5dec
  2. c5dec export chckComp3R5 3R5 -p ALC_COMP.1: ALC_COMP.1 does not exist in CCv3.1R5, thus an error message is shown.

Child links: TRB-018

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome The expected outcomes described in the steps are observed
verification_method Test
release beta

53 Export to spreadsheet an evaluation checklists with some assurance components of CCv2022R1 from the CLI TST-053

Test steps

  1. Run c5dec export chckComp2022 2022R1 -p ACO_REL.1 ALC_COMP.1

Child links: TRB-019

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome An evaluation checklists for components ACO_REL.1 and ALC_COMP.1 of CCv2022R1 is created in a spreadsheet at $C5DEC_ROOT/c5dec
verification_method Test
release beta

54 Generate default parts of an ETR document TST-054

Test steps

  1. Run c5dec etr -h

    • Expected output: a help information about the 'etr' command is displayed

  2. Run c5dec etr

    • Expected output: the files Acronyms-table.md, CMC-analysis.qmd, DocStruct-table.md and Glossary-table.md are created at $C5DEC_ROOT/c5dec/assets/etr/output

Child links: TRB-020

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition c5dec has been successfully installed
expected_outcome The expected outcomes described in the steps are observed
verification_method Test
release beta

55 Generate specific parts of an ETR document TST-055

Test steps

  1. Run c5dec etr -f <assurance_class> with random families of the ALC class.
  2. Run c5dec etr -t Acronyms

Child links: TRB-021

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome (i)A file named <family>-analysis.qmd is created per each selected family, with the content corresponding to the evaluation checklist of the assurance family. (ii) The file Acronyms-table.md is created too. All the files are stored in the folder $C5DEC_ROOT/c5dec/assets/etr/output.
verification_method Test
release beta

56 Generate an ETR report template using the C5-DEC DocEngine TST-056

Test steps

  1. Open the folder $c5DEC_ROOT/c5dec/assets/etr/ in VS Code
  2. In a VS Code terminal run: quarto render etr_template/chapters/intro.qmd

Child links: TRB-022

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition The VS Code extension for Quarto must be installed
expected_outcome The following output is created at the folder $c5DEC_ROOT/c5dec/assets/etr/etr_template/_output: (i) Single-Evaluation-Technical-Report.pdf (ii) Single-Evaluation-Technical-Report.docx (iii) html files corresponding to the report in HTML format
verification_method Test
release beta

57 Generate parts of an ETR report with an exported evaluation checklist TST-057

Test steps

  1. Run poetry run c5dec export etrInput 3R5 -c ALC ADV
  2. Copy the output spreadsheet from $C5DEC_ROOT/c5dec to $C5DEC_ROOT/c5dec/assets/etr/
  3. Rename the file as etrInput-tst-057.xlsx (for simplicity)
  4. Open the file $C5DEC_ROOT/c5dec/assets/c5dec_params.yml
  5. Modify the file as follows and then save it:
  etr:
    eval-wu-id: "WU-ID"
    #eval-file-name: "etr-eval-checklist"
    #eval-wu-sheet: "WU"
    #eval-awi-sheet: "AWI"
    eval-file-name: "etrInput-tst-057"
  1. Run poetry run c5dec etr -f LCD

Child links: TRB-023, TRB-025

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
expected_outcome Generated Markdown files with data from ALC_LCD are stored at $C5DEC_ROOT/c5dec/assets/etr/output
verification_method Test
release beta

58 Publish the project assets in HTML and Markdown TST-058

Test steps

  1. Run c5dec publish -h: the help text is displayed
  2. Run c5dec publish: html files are generated at $C5DEC_ROOT/c5dec/docs/publish
  3. Delete the folder created in 3.
  4. Run: c5dec publish -f .html: html files are generated at $C5DEC_ROOT/c5dec/docs/publish
  5. Run: c5dec publish -f .md: markdown files are generated at $C5DEC_ROOT/c5dec/docs/publish

Child links: TRB-024

Attribute Value
platform Any of MacOS, Windows, GNU/Linux
precondition Assets have been created with Doorstop
expected_outcome The outcomes described in the test steps are observed
verification_method Test
release beta