Use vanilla Spark in Iceberg integration tests in CI #1986
Description
What is the problem the feature request solves?
The Iceberg CI tests currently build a custom Spark with Comet enabled, which is the approach we use when running the Spark SQL tests. This approach should not be needed for running the Iceberg tests. We should just be able to pass additional Spark configuration settings to enable Comet.
This could reduce the CI time by more than 30 minutes.
Describe the potential solution
No response
Additional context
No response