1 Starter report for persisted output verification TRP-001

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-001. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future execution logs and artifact listings.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-001 Verify persisted output locations for VECTOR-Code artifacts

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

2 Starter report for language detection verification TRP-002

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with cloc installed.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-002. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future console output and sample project evidence.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-002 Verify supported language detection

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

3 Starter report for CodeQL database verification TRP-003

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with CodeQL installed.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-003. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future database directory listings and console logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-003 Verify CodeQL database creation

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

4 Starter report for SARIF generation verification TRP-004

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with query packs available.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-004. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future SARIF artifacts and query logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-004 Verify SARIF generation from CodeQL queries

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

5 Starter report for SARIF-to-CBOM verification TRP-005

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with cryptobom installed.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-005. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future CBOM output files and conversion logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-005 Verify SARIF-to-CBOM conversion

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

6 Starter report for raw TLS scan verification TRP-006

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with reachable TLS endpoint.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-006. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future raw TLS scan output and console logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-006 Verify raw TLS scan output generation

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

7 Starter report for TLS CBOM verification TRP-007

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with TLS mapping data present.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-007. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future CBOM output and mapped-component evidence.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-007 Verify TLS CBOM decomposition

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

8 Starter report for raw SSH scan verification TRP-008

Relevant test environment and configuration details

Planned environment: GNU/Linux workstation or dev container with reachable SSH endpoint.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-008. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future raw SSH scan output and console logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-008 Verify raw SSH scan output generation

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

9 Starter report for runtime assumption verification TRP-009

Relevant test environment and configuration details

Planned environment: valid and invalid Linux runtime layouts.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-009. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future environment setup records and failure logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-009 Verify Linux runtime assumptions and tool-path checks

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

10 Starter report for safe failure verification TRP-010

Relevant test environment and configuration details

Planned environment: invalid-input and missing-tool scenarios.
Execution status: deferred pending controlled test execution.

Test execution results

Test case step 1: Pending execution.

0 = flawless

Defect summary description

This starter report reserves the execution record for TCS-010. Formal execution evidence has not yet been collected in this repository.

Text execution evidence

Pending future validation-failure logs.

Comments

All three steps remain not executed in this initial baseline.

Parent links: TCS-010 Verify safe failure behavior

Attribute	Value
test-date	2026-04-11
tester	Pending execution
defect-category	0 = flawless
passed-steps	0
failed-steps	0
not-executed-steps	3
release-version	0.1
verification-method	T

11 Test report for algorithm classification TRP-011

Relevant test environment and configuration details

Environment: tor/vector_score/ within the dev container (Debian GNU/Linux 11, Python 3.11.0).
Test file: tests/test_algorithm_classifier.py
Test runner: poetry run pytest tests/test_algorithm_classifier.py -v
Execution date: 2026-06-03

Test execution results

Test case step 1: All 33 tests in test_algorithm_classifier.py passed — 0 = flawless

Selected evidence:

test_rsa_is_quantum_vulnerable → PASSED
test_ecdhe_is_quantum_vulnerable → PASSED
test_mlkem768_is_non_hybrid → PASSED
test_sntrup761_is_non_hybrid → PASSED
test_aes256_gcm_is_quantum_safe → PASSED
test_chacha20_poly1305_is_quantum_safe → PASSED
test_des_is_classically_deprecated → PASSED
test_md5_is_classically_deprecated → PASSED
test_secp256r1mlkem768_is_hybrid → PASSED
test_x25519mlkem768_is_hybrid → PASSED
test_unknown_algorithm_returns_unknown → PASSED

Full test run: 33 passed, 0 failed, 0 errors in less than a minute.

0 = flawless

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawless

Text execution evidence

See linked files (if any), e.g., screenshots, logs, etc.

Guide

Defect category: 0 = flawless; 1 = insignificant defect; 2 = minor defect; 3 = major defect; 4 = critical defect
Verification method (VM): Test (T), Review of design (R), Inspection (I), Analysis (A)

Parent links: TCS-011 Verify quantum risk classification

Attribute	Value
test-date	2026-06-03
tester	GDU
defect-category	0 = flawless
passed-steps	33
failed-steps	0
not-executed-steps	0
release-version	0.2
verification-method	T

12 Test report for CBOM annotated with pqcmat properties TRP-012

Relevant test environment and configuration details

Environment: vector container in VS Code with dev container extension
Test file: tests/test_cbom_scorer.py

Test execution results

Successful tests execution

0 = flawless: All the tests in test_cbom_scorer.py passed showing the output:

============================ test session starts ============================
platform linux -- Python 3.11.0, pytest-7.4.4, pluggy-1.6.0
rootdir: /home/vector/vector-project
plugins: anyio-4.13.0
collected 8 items

tests/test_cbom_scorer.py ........                                    [100%]

============================= 8 passed in 0.05s =============================

Adequate test coverage

0 = flawless: The test class has methods to verify that score_cbom() covers the validation cases as follows:
test_scored_cbom_has_pqcmat_properties: score_cbom() annotates all assets of type algorithm with pqcmat:risk-classification, pqcmat:risk-score (with a value in the given set), pqcmat:rationale, and pqcmat:recommended-migration
test_metadata_has_scored_at_property: the metadata.properties entry in the scored output contains an element "pqcmat:scored-at"
test_non_algorithm_components_not_annotated: pqcmat properties are not added to components with cryptoProperties.assetType != algorithm
test_crypto_asset_type_handled: the type variant crypto-asset used in certain tools for labeling cryptographic assets is supported. The two test data files in test/data used in the unit tests cover both variants cryptographic-asset and crypto-asset.

In addition, the class has methods to validate that the original CBOM input is not mutated.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawless

Test execution evidence

See reported output.

Parent links: TCS-012 Verify that annotated CBOM contains 'pqcmat' risk properties

Attribute	Value
test-date	2026-05-08
tester	IVS
defect-category	0 = flawless
passed-steps	2
failed-steps	0
not-executed-steps	0
release-version	0.3
verification-method	T, I

13 Test report for unified CBOM creation verification TRP-013

Relevant test environment and configuration details

Software deviations: aligned with test case specification
Execution environment: vector container built and run with the docker build and run commands, using the dev.Dockerfile

Test execution results

Successful execution of CBOM creation

0 = flawless: The command finished with the expected message and displayed the CBOM output path:

vector@7bcc4839d5fc:~/vector-project$ vector code tests/data/mixed_repo/
Language detection
  Detected: Python (12.3%)
  Detected: C++ (26.3%)

Creating CodeQL databases
  Created: db-python
  Created: db-cpp

Running crypto queries
  Results generated: crypto-python.sarif
  Results generated: crypto-cpp.sarif

Generating CBOM
  CBOM file generated at: /home/vector/vector-project/tor/vector_code/output/cbom/crypto-combined-cbom.json

Completed successfully

Correct output files

0 = flawless: The output folder tor/vector_code/output/ contains the expected structure:
cbom/crypto-combined-cbom.json
databases/db-cpp/
databases/db-python/
results/crypto-cpp.sarif
results/crypto-python.sarif

Algorithms from both files included in CBOM file

0 = flawless: Grep counts on cbom/crypto-combined-cbom.json confirm detection of cryptographic assets from both source files:

$ grep -c '"location": "crypto_func.py"' cbom/crypto-combined-cbom.json
2
$ grep -c '"location": "crypto_utils.cpp"' cbom/crypto-combined-cbom.json
3

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawless

Test execution evidence

See output reported per step.

Parent links: TCS-013 Verify creation of unified CBOM

Attribute	Value
test-date	2026-06-18
tester	IVS
defect-category	0 = flawless
passed-steps	3
failed-steps	0
not-executed-steps	0
release-version	0.3
verification-method	T

14 Test report for Markdown risk report TRP-014

Relevant test environment and configuration details

Software deviations: aligned with test case specification
Execution environment: vector container built and run with the docker build and run commands, using the dev.Dockerfile

Test execution results

Successful execution of vector score on the combined CBOM from TCS-013

0 = flawless: The command showed the path of the generated risk report and reported 5 algorithm components scored:

(vector-py3.11) vector@4fa3a2b8fcf8:~/vector-project$ vector score tor/vector_code/output/cbom/crypto-combined-cbom.json
Scoring tor/vector_code/output/cbom/crypto-combined-cbom.json
  Algorithm components scored: 5
  Annotated CBOM written to: tor/vector_code/output/cbom/crypto-combined-cbom_scored.json
  Risk report written to:    tor/vector_code/output/cbom/crypto-combined-cbom_risk_report.md

Both output files created

0 = flawless: Both paths are present after execution:

(vector-py3.11) vector@4fa3a2b8fcf8:~/vector-project$ ls tor/vector_code/output/cbom
crypto-combined-cbom.json
crypto-combined-cbom_risk_report.md
crypto-combined-cbom_scored.json

Top-heading structure with correct information

0 = flawless: The report's heading shows as target name the application name given to vector code and includes the timestamp and VECTOR-Score version

# Quantum-threat and cryptography risk report — application

**Scored at:** 2026-06-19T16:19:23.005955+00:00
**VECTOR-Score version:** 0.1
**Algorithm components scored:** 5

Summary table present with correct columns

0 = flawless: The "Summary" section contains a table with "Classification", "Risk score", and "Count" columns, listing 4 algorithms under the Quantum-Safe classification and 1 under Unknown:

## Summary

| Classification | Risk score | Count |
| --- | --- | --- |
| Quantum-Safe | None | 4 |
| Unknown / Unclassified | High | 1 |

The output in step 1 reported 5 algorithms being scored, which corresponds to the sum of the algorithms in the risk report.

Per-classification section with required columns

0 = flawless: The report contains a table with 3 AES and one SHA algorithms under the section Quantum-Safe and an "unknown" algorithm under the section Unknown/unclassified, which is consistent with the classification reported in the summary. The table has the 5 required columns plus a "Source locations" column.

Source file paths and line numbers present in the report

0 = flawless: The "Source locations" column displays the name of a source file and a line; the reported lines are:
crypto_utils.cpp L8
crypto_utils.cpp L5
crypto_utils.cpp L13
crypto_func.py L3
crypto_func.py L6

Normative references section present

0 = flawless: The Normative references section is present and lists standards cited for the scored algorithms, however, it includes more references than those that appear in the "Rationale". They are relevant though, probably coming from the catalog.

Defect summary description

Defect-free test execution, i.e., defect category: 0 = flawless

Test execution evidence

See output reported per step.

Comments

While having the source code location of the algorithms is useful, it would be more informative to have the full path. To avoid making the column look crowded, the path of the source code repository could be added to the top section.

If obtaining the path of the source code repository is not straight forward, a note on possible locations could be added, e.g., depending on the location of the output repository if the CBOM file is read from there.

Parent links: TCS-014 Verify Markdown risk report structure and source locations

Attribute	Value
test-date	2026-06-19
tester	IVS
defect-category	0 = flawless
passed-steps	7
failed-steps	0
not-executed-steps	0
release-version	0.3
verification-method	T