Skip to content

Conversation

@pietern
Copy link
Contributor

@pietern pietern commented Nov 3, 2025

Changes

See https://github.com/databricks/databricks-sdk-go/releases/tag/v0.89.0

This change removes previously added flags to the jobs create and pipelines create commands. These flags were not part of these commands before and were added by accident (in #3769). Because the payload for these commands is complex, they take a --json flag instead.

// Version could v0.0.0-dev+21e1aacf518a or just v0.0.0-dev (the latter is currently the case on Windows)
devVersionRegex = regexp.MustCompile(`0\.0\.0-dev(\+[a-f0-9]{10,16})?`)
// Matches databricks-sdk-go/0.90.0
sdkVersionRegex = regexp.MustCompile(`databricks-sdk-go/[0-9]+\.[0-9]+\.[0-9]+`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this means fewer diffs in the acceptance tests where recorded requests from Terraform show up with an older/newer SDK version until the TF dep is updated.

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Nov 3, 2025

Run: 19062766909

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip
🔄​ aws linux 3 1 324 591
💚​ aws windows 1 1 327 590
💚​ aws-ucws linux 1 1 449 486
💚​ aws-ucws windows 1 1 450 485
💚​ azure linux 1 1 326 590
💚​ azure windows 1 1 327 589
💚​ azure-ucws linux 1 1 445 485
💚​ azure-ucws windows 1 1 446 484
💚​ gcp linux 1 1 325 592
🔄​ gcp windows 8 1 1 318 591
11 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
TestAccept 🔄​f 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
TestAccept/bundle/destroy/jobs-and-pipeline ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/allow-duplicate-names ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/allow-duplicate-names/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/auto-approve ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/auto-approve/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/run/app-with-job 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
TestAccept/bundle/templates/default-python/integration_classic 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.11 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.12 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p


cmd.Flags().Var(&createJson, "json", `either inline JSON string or @path/to/file.json with request body`)

cmd.Flags().BoolVar(&createReq.AllowDuplicateNames, "allow-duplicate-names", createReq.AllowDuplicateNames, `If false, deployment will fail if name conflicts with that of another pipeline.`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be quite a big of backward incompatible change, shall we call it out at least in release notes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for flagging! I missed it when eyeballing the diff.

I'll investigate to see what happened to the field.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see it's not just the flag but all flags...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were added by mistake in #3769.

It was corrected in a later SDK release and that's now picked up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These flags were never intended to be added. Both the jobs create and pipeline create commands should always take a --json because the payloads are too complex to be specified via flags.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I included this in the PR summary and added a changelog entry.

@pietern pietern temporarily deployed to test-trigger-is November 4, 2025 08:42 — with GitHub Actions Inactive
@pietern pietern added this pull request to the merge queue Nov 4, 2025
Merged via the queue into main with commit 0b4f456 Nov 4, 2025
19 checks passed
@pietern pietern deleted the sdk-bump branch November 4, 2025 11:00
deco-sdk-tagging bot added a commit that referenced this pull request Nov 5, 2025
## Release v0.276.0

### CLI
* Remove previously added flags from the `jobs create` and `pipelines create` commands. ([#3870](#3870))

### Bundles
* Updated the default-python template to follow the Lakeflow conventions: pipelines as source files, pyproject.toml ([#3712](#3712)).
* Fix a permissions bug adding second IS\_OWNER and causing "The job must have exactly one owner." error. Introduced in 0.274.0. ([#3850](#3850))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants