Skip to content

Commit 1ff9340

Browse files
committed
[SPARK-54813] Remove cores configs from tests/e2e/watched-namespaces/spark-example.yaml
### What changes were proposed in this pull request? This PR aims to simplify `tests/e2e/watched-namespaces/spark-example.yaml` by removing irrelevant `spark.kubernetes.*.request.cores` configs and adding relevant `Spark Operator` configuration explicitly. ### Why are the changes needed? The main consumer of CPU resources is `Spark Operator` instead of `Spark Application`. We had better explicitly set `Spark Operator` CPU core request. ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #443 from dongjoon-hyun/SPARK-54813. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent c2b9a0a commit 1ff9340

File tree

2 files changed

+7
-2
lines changed

2 files changed

+7
-2
lines changed

tests/e2e/helm/dynamic-config-values.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,13 @@ workloadResources:
2424
roleBinding:
2525
create: true
2626

27+
operatorDeployment:
28+
operatorPod:
29+
operatorContainer:
30+
resources:
31+
requests:
32+
cpu: "0.1"
33+
2734
operatorConfiguration:
2835
dynamicConfig:
2936
enable: true

tests/e2e/watched-namespaces/spark-example.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,5 @@ spec:
2727
spark.executor.instances: "1"
2828
spark.kubernetes.container.image: "apache/spark:{{SPARK_VERSION}}-scala"
2929
spark.kubernetes.authenticate.driver.serviceAccountName: "spark"
30-
spark.kubernetes.driver.request.cores: "0.5"
31-
spark.kubernetes.executor.request.cores: "0.5"
3230
runtimeVersions:
3331
sparkVersion: 4.1.0

0 commit comments

Comments
 (0)