Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 182 additions & 0 deletions content/manuals/compose/compose-sdk.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

request: some more usage examples. Maybe take inspiration from the engine sdk examples page: https://deploy-preview-23751--docsdocker.netlify.app/reference/api/engine/sdk/examples/

Showing a few common use cases is probably a good idea to help people get going.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIT the SDK functions are self-explanatory as there's a 1:1 match with compose commands

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could say that about the engine SDK too but I still think examples are useful

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong, in many places the docker CLI relies on multiple API calls to achieve features people know well.

I'm not against examples, but I don't want to just write useless stuff like

// example: List Compose applications
applications, err := compose.List(ctx, ListOptions{})

I'd be happy to add more example once we get feedback on API usability from actual users

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah so when I say example, I don't mean showing function signatures. But we should be able to show some use cases beyond hello world. For example, using the EventProcessor. Or something with watch events, or custom logging, or even something more advanced like dependency visualization. Something to a) demo interesting capabilities, and b) show how these are meant to be used!

Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
---
description: Integrate Docker Compose directly into your applications with the Compose SDK.
keywords: docker compose sdk, compose api, Docker developer SDK
title: Using the Compose SDK
linkTitle: Compose SDK
weight: 60
params:
sidebar:
badge:
color: green
text: New
---

{{< summary-bar feature_name="Compose SDK" >}}

The `docker/compose` package can be used as a Go library by third-party applications to programmatically manage
containerized applications defined in Compose files. This SDK provides a comprehensive API that lets you
integrate Compose functionality directly into your applications, allowing you to load, validate, and manage
multi-container environments without relying on the Compose CLI.

Whether you need to orchestrate containers as part of
a deployment pipeline, build custom management tools, or embed container orchestration into your application, the
Compose SDK offers the same powerful capabilities that drive the Docker Compose command-line tool.

## Set up the SDK

To get started, create an SDK instance using the `NewComposeService()` function, which initializes a service with the
necessary configuration to interact with the Docker daemon and manage Compose projects. This service instance provides
methods for all core Compose operations including creating, starting, stopping, and removing containers, as well as
loading and validating Compose files. The service handles the underlying Docker API interactions and resource
management, allowing you to focus on your application logic.

### Requirements

Before using the SDK, make sure you're using a compatible version of the Docker CLI.

```go
require (
github.com/docker/cli v28.5.2+incompatible
)
```

Docker CLI version 29.0.0 and later depends on the new `github.com/moby/moby` module, whereas Docker Compose v5 currently depends on `github.com/docker/docker`. This means you need to pin `docker/cli v28.5.2+incompatible` to ensure compatibility and avoid build errors.

### Example usage

Here's a basic example demonstrating how to load a Compose project and start the services:

```go
package main

import (
"context"
"log"

"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/flags"
"github.com/docker/compose/v5/pkg/api"
"github.com/docker/compose/v5/pkg/compose"
)

func main() {
ctx := context.Background()

dockerCLI, err := command.NewDockerCli()
if err != nil {
log.Fatalf("Failed to create docker CLI: %v", err)
}
err = dockerCLI.Initialize(&flags.ClientOptions{})
if err != nil {
log.Fatalf("Failed to initialize docker CLI: %v", err)
}

// Create a new Compose service instance
service, err := compose.NewComposeService(dockerCLI)
if err != nil {
log.Fatalf("Failed to create compose service: %v", err)
}

// Load the Compose project from a compose file
project, err := service.LoadProject(ctx, api.ProjectLoadOptions{
ConfigPaths: []string{"compose.yaml"},
ProjectName: "my-app",
})
if err != nil {
log.Fatalf("Failed to load project: %v", err)
}

// Start the services defined in the Compose file
err = service.Up(ctx, project, api.UpOptions{
Create: api.CreateOptions{},
Start: api.StartOptions{},
})
if err != nil {
log.Fatalf("Failed to start services: %v", err)
}

log.Printf("Successfully started project: %s", project.Name)
}
```

This example demonstrates the core workflow - creating a service instance, loading a project from a Compose file, and
starting the services. The SDK provides many additional operations for managing the lifecycle of your containerized
application.

## Customizing the SDK

The `NewComposeService()` function accepts optional `compose.Option` parameters to customize the SDK behavior. These
options allow you to configure I/O streams, concurrency limits, dry-run mode, and other advanced features.

```go
// Create a custom output buffer to capture logs
var outputBuffer bytes.Buffer

// Create a compose service with custom options
service, err := compose.NewComposeService(dockerCLI,
compose.WithOutputStream(&outputBuffer), // Redirect output to custom writer
compose.WithErrorStream(os.Stderr), // Use stderr for errors
compose.WithMaxConcurrency(4), // Limit concurrent operations
compose.WithPrompt(compose.AlwaysOkPrompt()), // Auto-confirm all prompts
)
```

### Available options

- `WithOutputStream(io.Writer)`: Redirect standard output to a custom writer
- `WithErrorStream(io.Writer)`: Redirect error output to a custom writer
- `WithInputStream(io.Reader)`: Provide a custom input stream for interactive prompts
- `WithStreams(out, err, in)`: Set all I/O streams at once
- `WithMaxConcurrency(int)`: Limit the number of concurrent operations against the Docker API
- `WithPrompt(Prompt)`: Customize user confirmation behavior (use `AlwaysOkPrompt()` for non-interactive mode)
- `WithDryRun`: Run operations in dry-run mode without actually applying changes
- `WithContextInfo(api.ContextInfo)`: Set custom Docker context information
- `WithProxyConfig(map[string]string)`: Configure HTTP proxy settings for builds
- `WithEventProcessor(progress.EventProcessor)`: Receive progress events and operation notifications

These options provide fine-grained control over the SDK's behavior, making it suitable for various integration
scenarios including CLI tools, web services, automation scripts, and testing environments.

## Tracking operations with `EventProcessor`

The `EventProcessor` interface allows you to monitor Compose operations in real-time by receiving events about changes
applied to Docker resources such as images, containers, volumes, and networks. This is particularly useful for building
user interfaces, logging systems, or monitoring tools that need to track the progress of Compose operations.
Comment on lines +140 to +144
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repeating myself here but an example here would greatly help. It's quite opaque at the moment, hard to understand how this is meant to be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Roger, and I agree. Can I request that we get this PR merged now and I add and improve the docs in a separate PR? It's going to take me a bit more time to get that together and it'd be good to get what we have now live. I've addressed your other comments already


### Understanding `EventProcessor`

A Compose operation, such as `up`, `down`, `build`, performs a series of changes to Docker resources. The
`EventProcessor` receives notifications about these changes through three key methods:

- `Start(ctx, operation)`: Called when a Compose operation begins, for example `up`
- `On(events...)`: Called with progress events for individual resource changes, for example, container starting, image
being pulled
- `Done(operation, success)`: Called when the operation completes, indicating success or failure

Each event contains information about the resource being modified, its current status, and progress indicators when
applicable (such as download progress for image pulls).

### Event status types

Events report resource changes with the following status types:

- Working: Operation is in progress, for example, creating, starting, pulling
- Done: Operation completed successfully
- Warning: Operation completed with warnings
- Error: Operation failed

Common status text values include: `Creating`, `Created`, `Starting`, `Started`, `Running`, `Stopping`, `Stopped`,
`Removing`, `Removed`, `Building`, `Built`, `Pulling`, `Pulled`, and more.

### Built-in `EventProcessor` implementations

The SDK provides three ready-to-use `EventProcessor` implementations:

- `progress.NewTTYWriter(io.Writer)`: Renders an interactive terminal UI with progress bars and task lists
(similar to the Docker Compose CLI output)
- `progress.NewPlainWriter(io.Writer)`: Outputs simple text-based progress messages suitable for non-interactive
environments or log files
- `progress.NewJSONWriter()`: Render events as JSON objects
- `progress.NewQuietWriter()`: (Default) Silently processes events without producing any output

Using `EventProcessor`, a custom UI can be plugged into `docker/compose`.
2 changes: 1 addition & 1 deletion content/manuals/compose/releases/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
build:
render: never
title: Releases
weight: 70
weight: 80
---
2 changes: 1 addition & 1 deletion content/manuals/compose/releases/migrate.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
linkTitle: Migrate to Compose v2
Title: Migrate from Docker Compose v1 to v2
weight: 20
weight: 30
description: Step-by-step guidance to migrate from Compose v1 to v2, including syntax differences, environment handling, and CLI changes
keywords: migrate docker compose, upgrade docker compose v2, docker compose migration, docker compose v1 vs v2, docker compose CLI changes, docker-compose to docker compose
aliases:
Expand Down
Loading