Skip to content

Conversation

@kanterov
Copy link
Collaborator

@kanterov kanterov commented Apr 3, 2025

Changes

Add generated code for pipelines support introduced in #2574

@kanterov kanterov temporarily deployed to test-trigger-is April 3, 2025 09:52 — with GitHub Actions Inactive
@kanterov kanterov temporarily deployed to test-trigger-is April 3, 2025 14:24 — with GitHub Actions Inactive
@kanterov kanterov temporarily deployed to test-trigger-is April 4, 2025 13:55 — with GitHub Actions Inactive
@kanterov kanterov changed the title [Python] Add pipelines [Python] Add generated code for pipelines Apr 7, 2025
@kanterov kanterov marked this pull request as ready for review April 8, 2025 14:12
@kanterov kanterov temporarily deployed to test-trigger-is April 8, 2025 14:17 — with GitHub Actions Inactive
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target field is not included here, yet it is marked not deprecated in #2646.

How does this work?

@kanterov
Copy link
Collaborator Author

kanterov commented Apr 8, 2025

@pietern Good catch, the code implementing merging of JSON schema didn't work correctly after rebase. I've fixed it now

All necessary changes are in https://github.com/databricks/cli/tree/codegen-all, but it needs to be split into multiple PRs. It would be great if we could merge the first PR with codegen first and iterate on top of it.

@kanterov kanterov requested a review from pietern April 8, 2025 18:19
@pietern
Copy link
Contributor

pietern commented Apr 9, 2025

Thanks for clarifying. The first codegen PR has some small comments that should be easy to address + merge. Are you saying the tree you link has a bunch of changes stacked on top of what's in #2507?

@kanterov kanterov enabled auto-merge April 9, 2025 09:10
@kanterov
Copy link
Collaborator Author

kanterov commented Apr 9, 2025

@pietern there are a few changes needed:

  • enum changes that we are merging
  • adding deprecated annotations into the spec (there is draft PR)
  • adding "experimental" annotations into spec so it can be excluded from docs
  • handling newly added annotations, and a few tricks, e.g. excluding dataclass from docs if it's only used as experimental field

@kanterov kanterov temporarily deployed to test-trigger-is April 9, 2025 12:44 — with GitHub Actions Inactive
There are warnings because the same classes are included into jobs and pipeline packages.
We will fix it when we change code generation to vendor compute package into each resource
using it.

The output for docs already looks correct
@kanterov kanterov temporarily deployed to test-trigger-is April 9, 2025 12:57 — with GitHub Actions Inactive
@kanterov kanterov added this pull request to the merge queue Apr 9, 2025
Merged via the queue into main with commit 6083062 Apr 9, 2025
15 checks passed
@kanterov kanterov deleted the python-pipelines branch April 9, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants