Skip to content

Commit e50c7cf

Browse files
janmatzekjanmatzek
committed
feat: add gooddata-pipelines package
1 parent 9feeab8 commit e50c7cf

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+6247
-0
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.idea
2+
.vscode
23
*.iml
34
.env
45
.env.test

‎gooddata-pipelines/.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
**venv/*
2+
**/__pycache__/*
3+
**.env
4+
**.log
5+
dist/**

‎gooddata-pipelines/LICENSE.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# (C) 2025 GoodData Corporation
2+
BSD License
3+
4+
Copyright (c) 2023-2025, GoodData Corporation. All rights reserved.
5+
6+
Redistribution and use in source and binary forms, with or without modification, are permitted, provided that the following conditions are met:
7+
8+
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
9+
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
10+
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
11+
12+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎gooddata-pipelines/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# GoodData Pipelines
2+
3+
The `gooddata-pipelines` package provides a number of high level scripts to interact with and manage [GoodData](https://www.gooddata.com/).
4+
5+
At the moment, the package allows you to automate these features:
6+
7+
- Provisioning (creation, updtating and removal of resources) of
8+
- workspaces
9+
- users
10+
- user groups
11+
- user permissions
12+
- user data filters
13+
<!-- TODO: Backups, restores -->
14+
15+
<!-- TODO: link to documentation -->
16+
<!-- See [DOCUMENTATION](https://www.gooddata.com/docs/python-sdk/1.43.0) for more details. -->
17+
18+
## Requirements
19+
20+
- GoodData Cloud or GoodData.CN installation
21+
- Python 3.10 or newer
22+
23+
## Installation
24+
25+
Run the following command to install the `gooddata-pipelines` package on your system:
26+
27+
pip install gooddata-pipelines
28+
29+
## Example
30+
31+
This example illustrates how to use this package to manage GoodData Cloud workspaces. The provisioning script will ingest data which will be used as a source of truth to configure GoodData. The script will compare the source data with the current state of GoodData instance and will create, update and delete workspaces and user data filter settings to match the source data.
32+
33+
```python
34+
35+
# Import WorkspaceProvisioner and Workspace model.
36+
from gooddata_pipelines import Workspace, WorkspaceProvisioner
37+
38+
# Gather the raw data to be used by the provisioner.
39+
raw_data: list[dict] = [
40+
{
41+
"parent_id": "parent_workspace_id",
42+
"workspace_id": "workspace_id",
43+
"workspace_name": "Workspace Name",
44+
"workspace_data_filter_id": "wdf_id",
45+
"workspace_data_filter_values": ["value1", "value2"],
46+
}
47+
]
48+
49+
# Convert raw data to Workspace objects.
50+
data = [Workspace(**item) for item in raw_data]
51+
52+
# Create a WorkspaceProvisioner using your GoodData host name and token.
53+
host = "https://your-gooddata-host.com"
54+
token = "your_gooddata_token"
55+
provisioner = WorkspaceProvisioner.create(host=host, token=token)
56+
57+
# Provision the workspaces
58+
provisioner.provision(data)
59+
```
60+
61+
## Bugs & Requests
62+
63+
Please use the [GitHub issue tracker](https://github.com/gooddata/gooddata-python-sdk/issues) to submit bugs
64+
or request features.
65+
66+
## Changelog
67+
68+
See [Github releases](https://github.com/gooddata/gooddata-python-sdk/releases) for released versions
69+
and a list of changes.

‎gooddata-pipelines/TODO.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# TODO
2+
3+
A list of outstanding tasks, features, or technical debt to be addressed in this project.
4+
5+
## Pre-release
6+
7+
- [ ] License file
8+
9+
## Features
10+
11+
- [ ] Workspace backup
12+
- [ ] Workspace restore
13+
- [ ] Handle input from csv files [?]
14+
15+
## Refactoring / Debt
16+
17+
- [ ] Cleanup custom exception
18+
- [ ] Use objects to make `workspace_data_parses.py` more transparent
19+
- [ ] Use objects in API integration. Consider using existing Python SDK objects where possible, otherwise create pydantic models to use instead of `dict[str, Any]` stand-ins for json values
20+
- [ ] Improve test coverage. Write missing unit tests for legacy code (e.g., user data filters)
21+
22+
## Documentation
23+
24+
- [ ] Improve package README
25+
- [ ] Workspace provisioning
26+
- [ ] User provisioning
27+
- [ ] User group provisioning
28+
- [ ] Permission provisioning
29+
- [ ] User data filter provisioning

‎gooddata-pipelines/gooddata_pipelines/__init__.py‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# (C) 2025 GoodData Corporation
2+
3+
from ._version import __version__
4+
from .provisioning.entities.user_data_filters.models.udf_models import (
5+
UserDataFilter,
6+
)
7+
from .provisioning.entities.user_data_filters.user_data_filters import (
8+
UserDataFilterProvisioner,
9+
)
10+
from .provisioning.entities.users.models import Permission, User, UserGroup
11+
from .provisioning.entities.users.permissions import PermissionProvisioner
12+
from .provisioning.entities.users.user_groups import UserGroupProvisioner
13+
from .provisioning.entities.users.users import UserProvisioner
14+
from .provisioning.entities.workspaces.models import Workspace
15+
from .provisioning.entities.workspaces.workspace import WorkspaceProvisioner
16+
17+
__all__ = [
18+
"Workspace",
19+
"WorkspaceProvisioner",
20+
"User",
21+
"UserGroup",
22+
"Permission",
23+
"UserProvisioner",
24+
"UserGroupProvisioner",
25+
"PermissionProvisioner",
26+
"UserDataFilterProvisioner",
27+
"UserDataFilter",
28+
"__version__",
29+
]

‎gooddata-pipelines/gooddata_pipelines/_version.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# (C) 2025 GoodData Corporation
2+
from importlib import metadata
3+
4+
try:
5+
__version__ = metadata.version("gooddata-pipelines")
6+
except metadata.PackageNotFoundError:
7+
__version__ = "unknown-version"

‎gooddata-pipelines/gooddata_pipelines/api/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# (C) 2025 GoodData Corporation
2+
3+
from .gooddata_api_wrapper import GoodDataAPI
4+
5+
__all__ = ["GoodDataAPI"]

‎gooddata-pipelines/gooddata_pipelines/api/exceptions.py‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# (C) 2025 GoodData Corporation
2+
3+
"""Exception class for Panther operations.
4+
5+
This module defines the internally used `PantherException` class, which is used
6+
to handle exceptions that occur during operations related to the Panther SDK or
7+
GoodData Cloud API.
8+
"""
9+
10+
11+
class GoodDataApiException(Exception):
12+
"""Exception raised during Panther operations.
13+
14+
This exception is used to indicate errors that occur during operations
15+
related to interactions with the GoodData Python SDK or GoodData Cloud API.
16+
It can include additional context provided through keyword arguments.
17+
"""
18+
19+
def __init__(self, message: str, **kwargs: str) -> None:
20+
"""Raise a PantherException with a message and optional context.
21+
22+
Args:
23+
message (str): The error message describing the exception.
24+
**kwargs: Additional context for the exception, such as HTTP status,
25+
API endpoint, and HTTP method or any other relevant information.
26+
"""
27+
28+
super().__init__(message)
29+
self.error_message: str = message
30+
31+
# Set default values for attributes.
32+
# TODO: Consider if the defaults for these are still needed
33+
# - the values were necessary for log schema implementations, which
34+
# are not used anymore.
35+
self.http_status: str = "500 Internal Server Error"
36+
self.api_endpoint: str = "NA"
37+
self.http_method: str = "NA"
38+
39+
# Set attributes from kwargs.
40+
for key, value in kwargs.items():
41+
setattr(self, key, value)

0 commit comments

Comments
 (0)