Skip to content

USHIFT-6586: introduce tls-scanner test to optional suite #6266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
eslutsky wants to merge 1 commit into
base: main
Choose a base branch
from
Open

USHIFT-6586: introduce tls-scanner test to optional suite #6266

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions test/suites/optional/tls-scanner.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
*** Settings ***
Documentation Test tls-scanner tool with MicroShift host-based scanning.
... Clones openshift/tls-scanner, deploys the scanner job with
... scanner-job-microshift.yaml.template and SCAN_MODE=host,
... waits for completion, and collects results.
... See: https://github.com/openshift/tls-scanner

Library OperatingSystem
Library Process
Library String
Resource ../../resources/common.resource
Resource ../../resources/kubeconfig.resource
Resource ../../resources/oc.resource

Suite Setup Setup Suite With Namespace
Suite Teardown Teardown Suite With Namespace

Test Tags tls-scanner security optional


*** Variables ***
# Set by Suite Setup (common.resource / kubeconfig.resource):
${NAMESPACE} default
${KUBECONFIG} ${EMPTY}
# External: full tag of the scanner image to use (e.g. quay.io/my-org/tls-scanner:latest)
${SCANNER_IMAGE} quay.io/eslutsky/tls-scanner:latest
${TLS_SCANNER_REPO} https://github.com/eslutsky/tls-scanner
Comment on lines +26 to +27
Copy link

@coderabbitai coderabbitai bot Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Personal repo/image references should be updated to official openshift locations.

These appear to be development leftovers. For the PR to be merged, these should point to the official openshift repositories.

Proposed fix 
-${SCANNER_IMAGE}                quay.io/eslutsky/tls-scanner:latest
-${TLS_SCANNER_REPO}             https://github.com/eslutsky/tls-scanner
+${SCANNER_IMAGE}                ${EMPTY}
+${TLS_SCANNER_REPO}             https://github.com/openshift/tls-scanner

Note: Consider making SCANNER_IMAGE empty by default and requiring it to be passed externally, or use the official quay.io/openshift image path.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@test/suites/optional/tls-scanner.robot` around lines 26 - 27, The
SCANNER_IMAGE and TLS_SCANNER_REPO variables reference a personal quay/github
repo; update them to the official OpenShift locations (replace ${SCANNER_IMAGE}
value with the official quay.io/openshift/tls-scanner image or leave it empty
and require it to be supplied externally, and set ${TLS_SCANNER_REPO} to the
official OpenShift repo URL), adjusting the variable definitions in
tls-scanner.robot (look for SCANNER_IMAGE and TLS_SCANNER_REPO) so the PR no
longer points to personal/development artifacts.

${TLS_SCANNER_DIR} ${EMPTY}
${TLS_SCANNER_JOB_NAME} tls-scanner-job
${JOB_WAIT_TIMEOUT} 30m
${SCANNER_ARTIFACTS_DIR} ./artifacts
${CLUSTER_READER_MANIFEST} ./assets/tls-scanner/cluster-reader-clusterrole.yaml


*** Test Cases ***
TLS Scanner Host Scan Completes And Produces Artifacts
[Documentation] Clone tls-scanner, verify scanner image is available,
... deploy the scan job in host mode for MicroShift, wait for completion,
... and collect results (results.json, results.csv, scan.log).
[Setup] Run Keywords
... Check Required Scanner Variables
... Clone TLS Scanner Repo
... Ensure Cluster Reader Role Exists
Deploy TLS Scanner Job
Copy Scan Results Artifacts

[Teardown] Run Keywords
... Cleanup TLS Scanner Job


*** Keywords ***
Check Required Scanner Variables
[Documentation] Fail if SCANNER_IMAGE is not set.
Should Not Be Empty ${SCANNER_IMAGE}
... SCANNER_IMAGE must be set (full image tag, e.g. quay.io/my-org/tls-scanner:latest)

Ensure Cluster Reader Role Exists
[Documentation] Create cluster-reader ClusterRole for MicroShift (not shipped by default).
... deploy.sh expects this OpenShift role to exist for the scanner ServiceAccount.
Oc Apply -f ${CLUSTER_READER_MANIFEST}

Clone TLS Scanner Repo
[Documentation] Clone openshift/tls-scanner into a temporary directory.
${rand}= Generate Random String 8 [LOWER]
VAR ${workdir}= /tmp/tls-scanner-${rand}
Create Directory ${workdir}
VAR ${TLS_SCANNER_DIR}= ${workdir} scope=SUITE
${result}= Process.Run Process git clone --depth 1 ${TLS_SCANNER_REPO} .
... cwd=${TLS_SCANNER_DIR} shell=True timeout=120s
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0 msg=Failed to clone tls-scanner repo

Deploy TLS Scanner Job
[Documentation] Deploy the scanner job using MicroShift host template and SCAN_MODE=host.
${result}= Process.Run Process bash -x ./deploy.sh deploy
... cwd=${TLS_SCANNER_DIR}
... env:KUBECONFIG=${KUBECONFIG}
... env:SCANNER_IMAGE=${SCANNER_IMAGE}
... env:NAMESPACE=${NAMESPACE}
... env:JOB_TEMPLATE_FILE=scanner-job-microshift.yaml.template
... env:SCAN_MODE=host
... shell=True timeout=${JOB_WAIT_TIMEOUT}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0 msg=Failed to deploy tls-scanner job

Copy Scan Results Artifacts
[Documentation] Copy content of ${TLS_SCANNER_DIR}/artifacts to ${OUTPUTDIR}/tls-scanner-artifacts.
VAR ${dest}= ${OUTPUTDIR}/tls-scanner-artifacts
Create Directory ${dest}
${exists}= OperatingSystem.Directory Should Exist ${TLS_SCANNER_DIR}/artifacts
${files}= OperatingSystem.List Files In Directory ${TLS_SCANNER_DIR}/artifacts
${count}= Get Length ${files}
Should Be True ${count} > 0 msg=No artifacts produced by tls-scanner
FOR ${f} IN @{files}
Copy File ${TLS_SCANNER_DIR}/artifacts/${f} ${dest}/
END
Log Copied scan results to ${dest}/

Cleanup TLS Scanner Job
[Documentation] Remove the scanner job and RBAC via deploy.sh cleanup.
${result}= Run Keyword And Ignore Error Process.Run Process ./deploy.sh cleanup
... cwd=${TLS_SCANNER_DIR}
... env:KUBECONFIG=${KUBECONFIG}
... env:NAMESPACE=${NAMESPACE}
... shell=True timeout=60s
IF "${result[0]}" == "PASS" Log TLS scanner job cleanup completed
Remove Directory ${TLS_SCANNER_DIR} recursive=True
Comment on lines +101 to +109
Copy link

@coderabbitai coderabbitai bot Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Guard teardown against empty ${TLS_SCANNER_DIR}.

If setup fails before setting the suite variable, Remove Directory can error out during teardown. Make it best‑effort and conditional.

🔧 Proposed update 
-    Remove Directory    ${TLS_SCANNER_DIR}    recursive=True
+    IF    '${TLS_SCANNER_DIR}' != ''
+        Run Keyword And Ignore Error    Remove Directory    ${TLS_SCANNER_DIR}    recursive=True
+    END
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@test/suites/optional/tls-scanner.robot` around lines 101 - 109, The teardown
currently always calls Remove Directory on ${TLS_SCANNER_DIR}, which will error
if the variable is unset/empty; update the "Cleanup TLS Scanner Job" section to
guard the removal by checking the variable first (e.g., use IF/Run Keyword If to
test that ${TLS_SCANNER_DIR} is defined/non-empty and not just whitespace) and
only call Remove Directory when the check passes, keeping the existing Run
Keyword And Ignore Error call for deploy.sh and preserving recursive=True
behavior when removing the directory.