Skip to content

Conversation

@kanterov
Copy link
Collaborator

Changes

Add Python code generated from databricks bundle schema

Tests

Add a few tests for special cases in the generated code

@kanterov kanterov temporarily deployed to test-trigger-is March 13, 2025 11:58 — with GitHub Actions Inactive
@kanterov kanterov marked this pull request as ready for review March 13, 2025 12:26
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things:

  • Are you planning to move the codegen into this repository as well?
  • The distinction between the "jobs" and "compute" package will go away in the next version of the SDKs. We should plan for the migration path.

@kanterov
Copy link
Collaborator Author

kanterov commented Mar 17, 2025

@pietern

  1. Yes
  2. We already don't export any types from "compute" package. All necessary types are re-exported from "jobs" package, so I can write:
from databricks.bundles.jobs import ClusterSpec, PyPiLibrary, ...

We can manually add overrides to codegen to generate code in the "jobs" package, but then it will introduce divergence from the bundle schema. I think we should update the bundle schema, and codegen will naturally follow. Because of exports, it isn't going to be a breaking change. The only thing that will change is the __module__ value on classes.

@kanterov kanterov added this pull request to the merge queue Mar 17, 2025
Merged via the queue into main with commit 7c16044 Mar 17, 2025
14 checks passed
@kanterov kanterov deleted the add-databricks-bundles-py branch March 17, 2025 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants