Skip to content

Conversation

@kanterov
Copy link
Collaborator

@kanterov kanterov commented Mar 17, 2025

Changes

Add the code generator from the bundle schema to the Python code. Currently, it's only enabled for the "jobs" package.

There is a new GitHub action to verify that codegen is always up-to-date whenever we update bundle schema.

Depends on #2508

Tests

  • A few unit tests
  • Primarily tested by diffing output

@kanterov kanterov force-pushed the kanterov/python-codegen branch 3 times, most recently from 20b87ec to 5891b08 Compare March 17, 2025 15:26
@kanterov kanterov marked this pull request as ready for review April 3, 2025 09:34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is called openapi.py, but it looks like this exclusively depends on the JSON schema.

Is that correct?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, it was implementing OpenAPI until we introduced JSON schema into CLI, let me rename it.

@kanterov kanterov force-pushed the kanterov/python-codegen branch from 5891b08 to 977df62 Compare April 9, 2025 14:55
@kanterov kanterov requested a review from pietern April 10, 2025 11:33
@kanterov
Copy link
Collaborator Author

@pietern addressed your comments, please take a look again

Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please TAL at the comments.

#
# We can't load all types because of errors while loading some of them.
LOADED_NAMESPACES = [
"compute",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compute namespace is for job clusters, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's for job clusters and pipeline cluster configuration

# TODO block PR if this fails once diffs are fixed
# exit 1
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the tests run on push?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a little bit of a problem, if codegen itself crashed CI job will pass now, let me fix it

@github-actions
Copy link

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 2507
  • Commit SHA: 1a6449e2ff3b79911554a1cb6479e3d97e61c7c5

Checks will be approved automatically on success.

@pietern
Copy link
Contributor

pietern commented Apr 15, 2025

@kanterov Is this ready?

@kanterov kanterov enabled auto-merge April 15, 2025 09:54
@kanterov kanterov added this pull request to the merge queue Apr 15, 2025
Merged via the queue into databricks:main with commit 77ed564 Apr 15, 2025
16 checks passed
@kanterov kanterov deleted the kanterov/python-codegen branch April 15, 2025 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants