-
Notifications
You must be signed in to change notification settings - Fork 4.8k
CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: automatic mode |
|
@ropatil010: This pull request references CNTRLPLANE-2196 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/pipeline required |
|
Scheduling required tests: |
|
/retest |
|
Please ignore this test, I am just mis-using your PR to capture the test logs on an innocent change. |
|
/test e2e-gcp-ovn |
|
@ropatil010: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/lgtm |
|
/verified by CI runs |
|
@ropatil010: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
| binaryPath: "/usr/bin/oc-tests-ext.gz", | ||
| }, | ||
| { | ||
| imageTag: "cluster-authentication-operator", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are you sure that existing e2e tests for cluster-authentication-operator are executed in serial ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(^ not strictly related to the changes introduced in this PR :) )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @p0lyn0mial
Tried checking the logs, understanding the framework. This is what i got.
-
Tests run serially by default in Go. You only need to explicitly opt-in to parallel execution
-
So currently we are executing cases from test/e2e directory by using Ginkgo framework to run the tests in serial way.
-
As per our OTE implementation we have assigned names like
[sig-auth] authentication operator [Operator][Certs][Serial] TestRouterCerts
[sig-auth] authentication operator [Operator][Routes][Serial] TestCustomRouterCerts
[sig-auth] authentication operator [OIDC][Serial] TestGitLabAsOIDCPasswordGrantCheck
[sig-auth] authentication operator [OIDC][Serial] TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync
[sig-auth] authentication operator [Templates][Serial] TestTemplatesConfig
[sig-auth] authentication operator [Tokens][Serial] TestTokenInactivityTimeout
-
OTE suite configuration with parallelism: 1, Tests run sequentially (serially), not concurrently
-
Dedicated serial CI job in OpenShift's test infrastructure in PR: https://github.com/openshift/release/pull/73999/changes
-
The tests in test/e2e/ modify cluster-wide global state (OAuth config, routes, templates, etc.). If they ran in parallel the test will fail.
Execution logs not from my PR
https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-authentication-operator/829/pull-ci-openshift-cluster-authentic
ation-operator-master-e2e-operator/2016110000199438336/artifacts/e2e-operator/test/artifacts/junit_report.xml
ANALYZING ACTUAL JUNIT XML FROM PR #829
========================================
TEST DURATIONS FROM junit_report.xml:
- TestCustomRouterCerts: 42.83s
- TestGitLabAsOIDCPasswordGrantCheck: 291.25s
- TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync: 186.63s
- TestRouterCerts: 2.21s
- TestTemplatesConfig: 70.34s
- TestTokenInactivityTimeout: 2202.64s
└─ unset-inactivity-timeout-client-timeout: 610.57s
└─ with-client-timeout: 430.72s
└─ with-inactivity-timeout: 730.90s
└─ without-inactivity-timeout: 430.43s
CALCULATION:
-----------
Sum of individual test times:
42.83 + 291.25 + 186.63 + 2.21 + 70.34 + 2202.64 = 2795.90s
Total suite time reported in XML:
2796.978s
Difference: 2796.978 - 2795.90 = 1.078s (just overhead)
ANALYSIS:
---------
If PARALLEL: Total would be ≈ longest test (2202.64s)
If SERIAL: Total would be ≈ sum of all tests (2795.90s)
Actual total: 2796.978s ≈ Sum of all tests ✓
CONCLUSION: Tests ran SERIALLY (one after another)
mathematical proof from the actual CI logs that tests ran serially, not in parallel!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @p0lyn0mial Summary related to the tests under path test/e2e:
Test Name: TestRouterCerts
Implementation File:Line: certs.go:31
Test Wrapper File: certs_test.go:7
Current Label: [Operator][Certs][Serial]
Analysis: 🚫 Serial - Modifies global IngressController, changes cluster config
────────────────────────────────────────
Test Name: TestCustomRouterCerts
Implementation File:Line: custom_route.go:33
Test Wrapper File: custom_route_test.go:7
Current Label: [Operator][Routes][Serial]
Analysis: 🚫 Serial - Modifies cluster Ingresses config, updates component routes
────────────────────────────────────────
Test Name: TestGitLabAsOIDCPasswordGrantCheck
Implementation File:Line: gitlab.go:15
Test Wrapper File: gitlab_test.go:7
Current Label: [OIDC][Serial]
Analysis: 🚫 Serial - Adds GitLab IDP, modifies cluster OAuth config
────────────────────────────────────────
Test Name: TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync
Implementation File:Line: keycloak.go:27
Test Wrapper File: keycloak_test.go:7
Current Label: [OIDC][Serial]
Analysis: 🚫 Serial - Adds Keycloak IDP, creates global users/groups, triggers operator rollout
────────────────────────────────────────
Test Name: TestTemplatesConfig
Implementation File:Line: templates.go:28
Test Wrapper File: templates_test.go:7
Current Label: [Templates][Serial]
Analysis: 🚫 Serial - Modifies OAuth templates, triggers operator rollout, waits for progressing
────────────────────────────────────────
Test Name: TestTokenInactivityTimeout
Implementation File:Line: tokentimeout.go:36
Test Wrapper File: tokentimeout_test.go:7
Current Label: [Tokens][Serial]
Analysis: 🚫 Serial - Modifies OAuth token config, creates global resources, triggers operator rollout
Can you PTAL and do let me know about your opinion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @p0lyn0mial
Sure, Thanks. TBH i have taken reference from working service-ca-operator repo
OTA binary registration: https://github.com/openshift/origin/blob/main/pkg/test/extensions/binary.go#L219
Makefile: https://github.com/openshift/service-ca-operator/blob/main/Makefile#L53
Add serial job: https://github.com/openshift/release/pull/72396/changes
suite name: https://github.com/openshift/service-ca-operator/blob/main/cmd/service-ca-operator-tests-ext/main.go#L76
- ✅ In your test binary (cluster-authentication-operator-tests-ext/main.go):
Parallelism: 1 // Already there!
https://github.com/openshift/cluster-authentication-operator/blob/master/cmd/cluster-authentication-operator-tests-ext/main.go#L66 - ✅ In Makefile (keep current):
test-e2e: GO_TEST_FLAGS += -count 1 # Prevents caching and use count 1
https://github.com/openshift/cluster-authentication-operator/blob/master/Makefile#L49 - ✅ In CI job config (PR #73999):
env:
TEST_SUITE: openshift/cluster-authentication-operator/operator/serial
When you run tests via the OTE binary:
./cluster-authentication-operator-tests-ext run-suite openshift/cluster-authentication-operator/operator/serial
The binary's Parallelism: 1 setting controls execution, NOT go test flags.
The Makefile's -p 1 is only relevant for:
- Running go test ./test/e2e/... directly (without OTE binary)
- That's why encryption tests use -p 1 (they run via go test directly)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, so, it looks like we will update the auth operator to execute the tests in parallel (xref: openshift/cluster-authentication-operator#833)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah agree, as per your suggestions here openshift/cluster-authentication-operator#833 (comment) i have updated to parallel. Will monitor the CI job once it gets merged.
|
this PR registers the ote binary for the auth operator which is ok /lgtm |
|
/assign @stbenjam |
|
/assign @bertinatto |
bertinatto
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/approve
bertinatto
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/approve
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bertinatto, gangwgr, p0lyn0mial, ropatil010 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Scheduling required tests: |
03700fa
into
openshift:main
Hi Team,
PTAL, Register the OTE binary for cluster-authentication operator.
/assign @wangke19 @gangwgr @sandeepknd