Skip to content

Update user docs #843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ggainey merged 1 commit into from
May 16, 2025
Merged

Update user docs #843

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/user/guides/_SUMMARY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
* [Setup your own PyPI](pypi.md)
* [Set up your own PyPI](pypi.md)
* [Sync from Remote Repositories](sync.md)
* [Upload and Manage Content](upload.md)
* [Publish and Host Python Content](publish.md)
59 changes: 34 additions & 25 deletions docs/user/guides/publish.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,20 @@ Distribution before it is consumable).

```
{
"pulp_href": "/pulp/api/v3/publications/python/pypi/cad6007d-7172-41d1-8c22-0ec95e1d242a/",
"pulp_created": "2021-03-09T04:30:16.686784Z",
"repository_version": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/1/",
"repository": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/",
"pulp_href": "/pulp/api/v3/publications/python/pypi/0196ba31-cd04-7aba-a7b4-71a98a976745/",
"prn": "prn:python.pythonpublication:0196ba31-cd04-7aba-a7b4-71a98a976745",
"pulp_created": "2025-05-10T12:35:48.103758Z",
"pulp_last_updated": "2025-05-10T12:35:48.205361Z",
"repository_version": "/pulp/api/v3/repositories/python/python/0196ba30-e15e-71ea-9867-33aeceb5a87e/versions/1/",
"repository": "/pulp/api/v3/repositories/python/python/0196ba30-e15e-71ea-9867-33aeceb5a87e/",
"distributions": []
}
```

## Host a Publication (Create a Distribution)

To host a publication, (which makes it consumable by `pip`), users create a distribution which
will serve the associated publication at `/pypi/<distribution.base_path>/`
will serve the associated publication at `${BASE_ADDR}/pypi/${DIST_BASE_PATH}/`.

=== "Run"

Expand All @@ -49,15 +51,22 @@ will serve the associated publication at `/pypi/<distribution.base_path>/`

```
{
"pulp_href": "/pulp/api/v3/distributions/python/pypi/4839c056-6f2b-46b9-ac5f-88eb8a7739a5/",
"pulp_created": "2021-03-09T04:36:48.289737Z",
"base_path": "foo",
"base_url": "/pypi/foo/",
"content_guard": null,
"pulp_labels": {},
"name": "foo",
"publication": "/pulp/api/v3/publications/python/pypi/a09111b1-6bce-43ac-aed7-2e8441c22704/"
}
"pulp_href": "/pulp/api/v3/distributions/python/pypi/0196ba32-0be6-7f85-bbb3-cf561a4e2d88/",
"prn": "prn:python.pythondistribution:0196ba32-0be6-7f85-bbb3-cf561a4e2d88",
"pulp_created": "2025-05-10T12:36:04.200189Z",
"pulp_last_updated": "2025-05-10T12:36:04.200232Z",
"base_path": "foo",
"base_url": "http://localhost:5001/pypi/foo/",
"content_guard": null,
"no_content_change_since": "2025-05-10T12:36:04.200232Z",
"hidden": false,
"pulp_labels": {},
"name": "foo",
"repository": null,
"publication": "/pulp/api/v3/publications/python/pypi/0196ba31-cd04-7aba-a7b4-71a98a976745/",
"allow_uploads": true,
"remote": null
}
```

## Automate Publication and Distribution
Expand All @@ -68,7 +77,7 @@ updated automatically when new repository versions are created.
```bash
# This configures the repository to produce new publications when a new version is created
pulp python repository update --name foo --autopublish
# This configures the distribution to be track the latest repository version for a given repository
# This configures the distribution to track the latest repository version for a given repository
pulp python distribution update --name foo --repository foo
```

Expand All @@ -77,7 +86,7 @@ pulp python distribution update --name foo --repository foo
Functionality may not work or may be incomplete. Also, backwards compatibility when upgrading
is not guaranteed.

## Enable Pull-Through Caching:
## Enable Pull-Through Caching

Only packages present in your repository will be available from your index, but adding a remote source to
your distribution will enable the pull-through cache feature. This feature allows you to install any package
Expand All @@ -101,28 +110,26 @@ pulp python distribution update --name foo --remote bar
The metadata and packages can now be retrieved from the distribution:

```bash
$ http $BASE_ADDR/pypi/foo/simple/
$ http $BASE_ADDR/pypi/foo/simple/shelf-reader/
http "${BASE_ADDR}/pypi/foo/simple/"
http "${BASE_ADDR}/pypi/foo/simple/shelf-reader/"
```

!!! note
When domains are enabled, it is necessary to include the domain name within the URL, like so:
`$BASE_ADDR/pypi/${DOMAIN_NAME}/foo/simple/`
`${BASE_ADDR}/pypi/${DOMAIN_NAME}/foo/simple/`

The content is also pip installable:

```bash
$ pip install --trusted-host localhost -i $BASE_ADDR/pypi/foo/simple/ shelf-reader
pip install --trusted-host localhost -i "${BASE_ADDR}/pypi/foo/simple/" shelf-reader
```

If you don't want to specify the distribution path every time, you can modify your `pip.conf`
file. See the [pip docs](https://pip.pypa.io/en/stable/user_guide/#configuration) for more
detail.:
If you don't want to specify the distribution path every time, you can modify your `pip.conf` file:

=== "Run"

```bash
$ cat pip.conf
cat pip.conf
```

=== "Output"
Expand All @@ -135,5 +142,7 @@ detail.:
The above configuration informs `pip` to install from `pulp`:

```bash
$ pip install --trusted-host localhost shelf-reader
pip install --trusted-host localhost shelf-reader
```

See the [pip docs](https://pip.pypa.io/en/stable/topics/configuration) for more details.
60 changes: 35 additions & 25 deletions docs/user/guides/pypi.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Setup your own PyPI:
# Set up your own PyPI

This section guides you through the quickest way to setup `pulp_python` to act as your very own
This section guides you through the quickest way to set up `pulp_python` to act as your very own
private `PyPI`.

## Create a Repository:
## Create a Repository

Repositories are the base objects `Pulp` uses to store and organize its content. They are automatically
versioned when content is added or deleted and allow for easy rollbacks to previous versions.
Expand All @@ -19,20 +19,22 @@ versioned when content is added or deleted and allow for easy rollbacks to previ

```
{
"pulp_href": "/pulp/api/v3/repositories/python/python/3fe0d204-217f-4250-8177-c83b30751fca/",
"pulp_created": "2021-06-02T14:54:53.387054Z",
"versions_href": "/pulp/api/v3/repositories/python/python/3fe0d204-217f-4250-8177-c83b30751fca/versions/",
"pulp_labels": {},
"latest_version_href": "/pulp/api/v3/repositories/python/python/3fe0d204-217f-4250-8177-c83b30751fca/versions/1/",
"name": "foo",
"description": null,
"retained_versions": null,
"remote": null,
"autopublish": false
}
"pulp_href": "/pulp/api/v3/repositories/python/python/0196ba29-52b9-7cf4-b12e-f3247f0eb3dc/",
"prn": "prn:python.pythonrepository:0196ba29-52b9-7cf4-b12e-f3247f0eb3dc",
"pulp_created": "2025-05-10T12:26:32.506906Z",
"pulp_last_updated": "2025-05-10T12:26:32.517333Z",
"versions_href": "/pulp/api/v3/repositories/python/python/0196ba29-52b9-7cf4-b12e-f3247f0eb3dc/versions/",
"pulp_labels": {},
"latest_version_href": "/pulp/api/v3/repositories/python/python/0196ba29-52b9-7cf4-b12e-f3247f0eb3dc/versions/0/",
"name": "foo",
"description": null,
"retain_repo_versions": null,
"remote": null,
"autopublish": false
}
```

## Create a Distribution:
## Create a Distribution

Distributions serve the content stored in repositories so that it can be used by tools like `pip`.

Expand All @@ -46,35 +48,43 @@ Distributions serve the content stored in repositories so that it can be used by

```
{
"pulp_href": "/pulp/api/v3/distributions/python/pypi/e8438593-fd40-4654-8577-65398833c331/",
"pulp_created": "2021-06-03T20:04:18.233230Z",
"pulp_href": "/pulp/api/v3/distributions/python/pypi/0196ba29-8f95-776b-a782-78d7838f3f9f/",
"prn": "prn:python.pythondistribution:0196ba29-8f95-776b-a782-78d7838f3f9f",
"pulp_created": "2025-05-10T12:26:48.086775Z",
"pulp_last_updated": "2025-05-10T12:26:48.086806Z",
"base_path": "my-pypi",
"base_url": "https://pulp3-source-fedora33.localhost.example.com/pypi/foo/",
"base_url": "http://localhost:5001/pypi/my-pypi/",
"content_guard": null,
"no_content_change_since": null,
"hidden": false,
"pulp_labels": {},
"name": "my-pypi",
"repository": "/pulp/api/v3/repositories/python/python/3fe0d204-217f-4250-8177-c83b30751fca/",
"repository": "/pulp/api/v3/repositories/python/python/0196ba29-52b9-7cf4-b12e-f3247f0eb3dc/",
"publication": null,
"allow_uploads": true
"allow_uploads": true,
"remote": null
}
```

## Upload and Install Packages:
## Upload and Install Packages

Packages can now be uploaded to the index using your favorite Python tool. The index url will be available
at `/pypi/<distribution.base_path>/simple/`.
at `${BASE_ADDR}/pypi/${DIST_BASE_PATH}/simple/`.

```bash
BASE_ADDR="http://localhost:5001"
PLUGIN_SOURCE="shelf-reader"
git clone https://github.com/asmacdo/shelf-reader.git
# Build custom package
python -m build $PLUGIN_SOURCE
python -m build "$PLUGIN_SOURCE"
# Upload built package distributions to my-pypi
twine upload --repository-url $BASE_ADDR/pypi/my-pypi/simple/ -u admin -p password "$PLUGIN_SOURCE"dist/*
twine upload --repository-url "${BASE_ADDR}/pypi/my-pypi/simple/" -u admin -p password "${PLUGIN_SOURCE}/dist/"*
```

Packages can then be installed using your favorite Python tool:

```bash
pip install --trusted-host localhost -i $BASE_ADDR/pypi/my-pypi/simple/ shelf-reader
pip install --trusted-host localhost -i "${BASE_ADDR}/pypi/my-pypi/simple/" "$PLUGIN_SOURCE"
```

Now you have a fully operational Python package index. Check out the other workflows to see more features of
Expand Down
85 changes: 58 additions & 27 deletions docs/user/guides/sync.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
Users can populate their repositories with content from an external source like PyPI by syncing
their repository.


## Create a Repository

=== "Run"
Expand All @@ -17,21 +16,23 @@ their repository.

```
{
"pulp_href": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/",
"pulp_created": "2021-03-09T04:11:54.347921Z",
"versions_href": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/",
"pulp_labels": {},
"latest_version_href": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/0/",
"name": "foo",
"description": null,
"remote": null
}
"pulp_href": "/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/",
"prn": "prn:python.pythonrepository:0196ba2a-f353-736a-854c-2d415389a509",
"pulp_created": "2025-05-10T12:28:19.156941Z",
"pulp_last_updated": "2025-05-10T12:28:19.169190Z",
"versions_href": "/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/versions/",
"pulp_labels": {},
"latest_version_href": "/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/versions/0/",
"name": "foo",
"description": null,
"retain_repo_versions": null,
"remote": null,
"autopublish": false
}
```

Reference: [Python Repository Usage](site:pulp_python/restapi/#tag/Repositories:-Python)



## Create a Remote

Creating a remote object informs Pulp about an external content source. In this case, we will be
Expand All @@ -49,29 +50,56 @@ itself, a fixture, or even an instance of Pulp 2.

```
{
"pulp_href": "/pulp/api/v3/remotes/python/python/a9bb3a02-c7d2-4b2e-9b66-050a6c9b7cb3/",
"pulp_created": "2021-03-09T04:14:02.646835Z",
"pulp_href": "/pulp/api/v3/remotes/python/python/0196ba2b-1461-7d0d-99f6-5f75610abf71/",
"prn": "prn:python.pythonremote:0196ba2b-1461-7d0d-99f6-5f75610abf71",
"pulp_created": "2025-05-10T12:28:27.617672Z",
"pulp_last_updated": "2025-05-10T12:28:27.617697Z",
"name": "bar",
"url": "https://pypi.org/",
"ca_cert": null,
"client_cert": null,
"tls_validation": true,
"proxy_url": null,
"pulp_labels": {},
"pulp_last_updated": "2021-03-09T04:14:02.646845Z",
"download_concurrency": 10,
"download_concurrency": null,
"max_retries": null,
"policy": "on_demand",
"total_timeout": null,
"connect_timeout": null,
"sock_connect_timeout": null,
"sock_read_timeout": null,
"headers": null,
"rate_limit": null,
"hidden_fields": [
{
"name": "client_key",
"is_set": false
},
{
"name": "proxy_username",
"is_set": false
},
{
"name": "proxy_password",
"is_set": false
},
{
"name": "username",
"is_set": false
},
{
"name": "password",
"is_set": false
}
],
"includes": [
"shelf-reader"
],
"excludes": [],
"prereleases": true,
"package_types": [],
"keep_latest_packages": 0,
"exclude_platforms": []
}
```

Expand All @@ -80,7 +108,7 @@ Reference: [Python Remote Usage](site:pulp_python/restapi/#tag/Remotes:-Python)
## A More Complex Remote

If only the name of a project is specified, every distribution of every version of that project
will be synced. You can use the version_specifier field to ensure only distributions you care
will be synced. You can use the version specifier field to ensure only distributions you care
about will be synced:

```bash
Expand Down Expand Up @@ -115,22 +143,22 @@ You can also filter packages by their type, platform and amount synced through t
"exclude_platforms", and "keep_latest_packages" fields respectively, like so:

```bash
# Sync only sdist and bdist_wheel package types, exclude any packages built
# for windows and keep the five latest versions
pulp python remote create \
--name 'complex-filters' \
--url 'https://pypi.org/' \
--includes '["django"]' \
--package-types '["sdist", "bdist-wheel"]' # only sync sdist and bdist-wheel package types \
--exclude-platforms '["windows"]' # exclude any packages built for windows \
--keep-latest-packages 5 # keep the five latest versions
--package-types '["sdist", "bdist_wheel"]' \
--exclude-platforms '["windows"]' \
--keep-latest-packages 5
```

Reference: [Python Remote Usage](site:pulp_python/restapi/#tag/Remotes:-Python)



### Creating a remote to sync all of PyPI

A remote can be setup to sync all of PyPI by not specifying any included packages like so:
A remote can be set up to sync all of PyPI by not specifying any included packages, like so:

```bash
pulp python remote create \
Expand Down Expand Up @@ -173,22 +201,25 @@ sync with. You are telling pulp to fetch content from the remote and add to the

```
{
"pulp_href": "/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/1/",
"pulp_created": "2021-03-09T04:20:21.896132Z",
"pulp_href": "/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/versions/1/",
"prn": "prn:core.repositoryversion:0196ba2b-655c-7745-b10f-bdde15a941c6",
"pulp_created": "2025-05-10T12:28:48.349938Z",
"pulp_last_updated": "2025-05-10T12:28:49.031497Z",
"number": 1,
"repository": "/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/",
"base_version": null,
"content_summary": {
"added": {
"python.python": {
"count": 2,
"href": "/pulp/api/v3/content/python/packages/?repository_version_added=/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/1/"
"href": "/pulp/api/v3/content/python/packages/?repository_version_added=/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/versions/1/"
}
},
"removed": {},
"present": {
"python.python": {
"count": 2,
"href": "/pulp/api/v3/content/python/packages/?repository_version=/pulp/api/v3/repositories/python/python/8fbb24ee-dc91-44f4-a6ee-beec60aa542d/versions/1/"
"href": "/pulp/api/v3/content/python/packages/?repository_version=/pulp/api/v3/repositories/python/python/0196ba2a-f353-736a-854c-2d415389a509/versions/1/"
}
}
}
Expand Down
Loading