1 TRB-001
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-035 Run the GUI from the CLI with option --gui
| Attribute | Value |
|---|---|
| test_date | 14-07-2024 |
| tester | IVS |
| defect_category | 0 |
2 TRB-002
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-036 Run the GUI from the CLI with option -g
| Attribute | Value |
|---|---|
| test_date | 14-07-2024 |
| tester | IVS |
| defect_category | 0 |
3 TRB-003
The test execution for the test case (TST) referred in the parent link results in the following observations:
In the CCT user manual, the part concerning the [ETR creation] is confusing: - The description reads as if the user always needs to create an evaluation checklist before use this command. - The families (and classes) included in the default evaluation checklist could be clarified. - A short section describing a simple flow for the 'export'-'etr'-quarto integration would be very helpful.
Parent links: TST-037 The user manual contains guides for the functionality added in the Beta release
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 2 |
| defect_description | The text needs more clarity with respect on how to perform export and ETR integration |
3.1 TRB-031
Re-execution of test case TST-043 after a code update addressing the issues reported in the first testing campaign of the beta release.
Parent links: TST-037 The user manual contains guides for the functionality added in the Beta release
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The manual has been modified with the clarifications requested in TRB-003. |
4 TRB-004
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-038 Browse the CC database in the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | Adding a legend in the GUI to inform about the version of CC in use is recommended. |
5 TRB-005
Test execution results for the test case (TST) referred in the parent link. Evidence is provided in the files:
docs/specs/trb/assets/trb-005i-ev.pngdocs/specs/trb/assets/trb-005ii-ev.png
Parent links: TST-039 The CC browser displays the content of the selected items
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | When selecting a CC item from a dropdown list, clearing the dropdown lists of the child CC items would be very helpful for navigation. This is just a reminder as the developers are aware of the issue. |
6 TRB-006
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-040 Copy content in markdown from the CC browser text area
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
7 TRB-007
Test execution results for the test case (TST) referred in the parent link. Evidence pictures are provided in the files:
docs/specs/trb/assets/trb-007a-ev.pngdocs/specs/trb/assets/trb-007b-ev.png
Parent links: TST-041 Expand the text including information of all the descendents of the selected item
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
8 TRB-008
Test execution results for the test case (TST) referred in the parent link. The files and evaluation list are created in markdown and .xsls formats. Evidence pictures are provided in the files:
docs/specs/trb/assets/trb-008i-ev.pngdocs/specs/trb/assets/trb-008ii-ev.pngdocs/specs/trb/assets/trb-008iii-ev.png
Parent links: TST-042 Create spreadsheet evaluation checklist of an assurance class of CCv3.1R5 from the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | Suggestion: add some text in the GUI to inform the path where the exported file is created |
8.1 TRB-029
Re-execution of test case TST-042 after a code update addressing the issues reported in the first testing campaign of the beta release. This test execution covers the section Updates 18-07-2024 in the test case description.
Parent links: TST-042 Create spreadsheet evaluation checklist of an assurance class of CCv3.1R5 from the GUI
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The suggestion on TRB-008 has been addressed |
9 TRB-009
Test execution results for the test case (TST) referred in the parent link. The folders and files were successfully generated.
Parent links: TST-043 Create spreadsheet evaluation checklist of all classes of CCv3.1R5 from the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
9.1 TRB-030
Re-execution of test case TST-043 after a code update addressing the issues reported in the first testing campaign of the beta release. This test execution covers the section Updates 18-07-2024 in the test case description.
Parent links: TST-043 Create spreadsheet evaluation checklist of all classes of CCv3.1R5 from the GUI
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
10 TRB-010
Test execution results for the test case (TST) referred in the parent link. The evidence for this test is shown also in:
docs/specs/trb/assets/trb-008i-ev.pngdocs/specs/trb/assets/trb-008ii-ev.pngdocs/specs/trb/assets/trb-008iii-ev.png
Parent links: TST-044 Create spreadsheet evaluation checklist of an assurance class of CCv2022R1 from the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
11 TRB-011
Test execution results for the test case (TST) referred in the parent link. The folders and files were successfully generated. Evidence in the referred file.
docs/specs/trb/assets/trb-011iii-ev.png
Parent links: TST-045 Create spreadsheet evaluation checklist of all classes of CC2022R1 from the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
12 TRB-012
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-046 Navigate using the links in the GUI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
13 TRB-013
Test execution results for the test case (TST) referred in the parent link. Evidence in the path:
docs/specs/trb/assets/trb-013-ev.png
Parent links: TST-047 Export to spreadsheet an evaluation checklists with all the assurance classes of CCv3.1R5 from the CLI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 1 |
| defect_description | The file is correctly generated but the format of its name differs from the expected name format. |
| comments | This defect is associated to issue 46 |
13.1 TRB-028
Re-execution of test case TST-047 after a code update addressing the issues reported in the first testing campaign of the beta release.
Note: The output path of the spreadsheet has changed from
$C5DEC_ROOT/c5dec to $C5DEC_ROOT/c5dec/export/.
Parent links: TST-047 Export to spreadsheet an evaluation checklists with all the assurance classes of CCv3.1R5 from the CLI
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The defect reported in TRB-013 has been fixed. |
14 TRB-014
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-048 Export to spreadsheet an evaluation checklists with all the assurance classes of CC2022R1 from the CLI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 1 |
| defect_description | The file is correctly generated but the format of its name differs from the expected name format. |
| comments | This defect is associated to issue 46 |
14.1 TRB-027
Re-execution of test case TST-048 after a code update addressing the issues reported in the first testing campaign of the beta release.
Note: The output path of the spreadsheet has changed from
$C5DEC_ROOT/c5dec to $C5DEC_ROOT/c5dec/export/.
Evidence is provided in the referred file.
docs/specs/trb/assets/trb-027-ev.png
Parent links: TST-048 Export to spreadsheet an evaluation checklists with all the assurance classes of CC2022R1 from the CLI
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The defect reported in TRB-014 has been fixed. |
15 TRB-015
Test execution results for the test case (TST) referred in the parent link. The evidence output file is:
docs/specs/trb/assets/trb-015-ev.png
Parent links: TST-049 Export to spreadsheet an evaluation checklists with some assurance classes of CCv3.1R5 from the CLI
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 0 |
16 TRB-016
Test execution results for the test case (TST) referred in the parent link. The evidence output file is:
docs/specs/trb/assets/trb-016-ev.png
Parent links: TST-050 Export to spreadsheet an evaluation checklists with some assurance classes of CC2022R1 from the CLI
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
17 TRB-017
The test execution results for the test case (TST) referred in the parent link are as follows:
- When given an invalid version the CLI crashes ungracefully.
- OK
- When the input argument must be an assurance class, the error suggests that it must be an assurance component:
ERROR: Input components must be of type AComponent.
In steps 4 and 5 (invalid assurance component/class), the error message is shown as part of the error trace, which makes the output confusing:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/alab/c5dec/c5dec/frontend/cli/main.py", line 88, in run
success = function(args, os.getcwd(), parser.error)
File "/home/alab/c5dec/c5dec/frontend/cli/commands.py", line 143, in run_export
cct.ChecklistBuilder(checklist_name=args.name, cc_version=args.version).export_eval_checklist(class_id_vector=args.classes, component_id_vector=args.components)
File "/home/alab/c5dec/c5dec/core/cct.py", line 3752, in export_eval_checklist
component_id_vector = self.build_component_id_vector_for_classes(class_id_vector)
File "/home/alab/c5dec/c5dec/core/cct.py", line 3730, in build_component_id_vector_for_classes
cc_class = cc_index.get(cc_class_id)
File "/home/alab/c5dec/c5dec/core/cct.py", line 122, in get
raise KeyError(error_msg)
KeyError: 'Invalid Id ACO_RER!'
Parent links: TST-051 Export command: messages for invalid input
| Attribute | Value |
|---|---|
| test_date | 15-07-2024 |
| tester | IVS |
| defect_category | 3 |
| defect_description | The CLI crashes ungracefully (step 1). |
| comments | This defect has been reported in issue 47 |
17.1 TRB-026
Re-execution of test case TST-051 after a code update addressing the issues reported in the first testing campaign of the beta release.
Parent links: TST-051 Export command: messages for invalid input
| Attribute | Value |
|---|---|
| test_date | 18/07/2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The defects reported in TRB-017 have been fixed |
18 TRB-018
The test execution results for all the steps in the test case (TST) referred in the parent link were successful. Evidence in the files:
docs/specs/trb/assets/trb-018i-ev.pngdocs/specs/trb/assets/trb-018ii-ev.png
Parent links: TST-052 Export to spreadsheet an evaluation checklists with some assurance components of CCv3.1R5 from the CLI
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
19 TRB-019
Test execution results for the test case (TST) referred in the parent link. Evidence in the file:
docs/specs/trb/assets/trb-019-ev.png
Parent links: TST-053 Export to spreadsheet an evaluation checklists with some assurance components of CCv2022R1 from the CLI
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
20 TRB-020
Test execution results for the test case (TST) referred in the parent link.
docs/specs/trb/assets/trb-020-ev.png
Parent links: TST-054 Generate default parts of an ETR document
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
21 TRB-021
Test execution results for the test case (TST) referred in the parent link.
The files are generated, yet, many of them are empty as the default evaluation checklist was used.
Evidence is provided in the file referenced below. Note the generated file sksdj-analysis.qmd
docs/specs/trb/assets/trb-021-ev.png
Parent links: TST-055 Generate specific parts of an ETR document
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 1 |
| defect_description | Empty files are created even for invalid family names (see evidence file). A validation previous to generating the files would help. |
| comments | Indicate in the CCT user manual the default available families. UPDATE 18/07/2024: this issues is not considered a priority for the beta release as functionality does not get affected. |
22 TRB-022
Test execution results for the test case (TST) referred in the parent link. Evidence is provided in the file referenced below.
docs/specs/trb/assets/trb-022a-ev.pngdocs/specs/trb/assets/trb-022b-ev.png
Parent links: TST-056 Generate an ETR report template using the C5-DEC DocEngine
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
23 TRB-023
Test execution results for the test case (TST) referred in the parent link. At step 6, the following output message is obtained:
$ poetry run c5dec etr -f LCD
assets/etr/output
'dict' object has no attribute 'loc'
Parent links: TST-057 Generate parts of an ETR report with an exported evaluation checklist
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 3 |
| defect_description | The etr command used on a checklist previously generated with the export command does not generate an output |
| comments | Changing the default tab name in etrInput-tst-057.xsls tu "WU" (with eval-wu-sheet: "WU" uncommented in c5dec_params.yml) did not have a different effect. |
23.1 TRB-025
Re-execution of test case TST-057 after a code update addressing the issues reported in the first testing campaign of the beta release.
Notes:
-
The output path of the spreadsheet (step 2) has changed. Instead of
$C5DEC_ROOT/c5dec, now it is$C5DEC_ROOT/c5dec/export/. -
The parameters
eval-wu-sheetandeval-awu-sheetin step 5 do not need to be commented anymore.
Evidence is provided in the referred file.
docs/specs/trb/assets/trb-025-ev.png
Parent links: TST-057 Generate parts of an ETR report with an exported evaluation checklist
| Attribute | Value |
|---|---|
| test_date | 18-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | The defect reported in TRB-023 has been fixed |
24 TRB-024
Test execution results for the test case (TST) referred in the parent link.
Parent links: TST-058 Publish the project assets in HTML and Markdown
| Attribute | Value |
|---|---|
| test_date | 16-07-2024 |
| tester | IVS |
| defect_category | 0 |
| comments | A message informing of the output path is needed. |