You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
:page-description: Learn about Chunk by CircleCI and how it can automatically identify and fix flaky tests in your CI/CD pipelines.
5
5
:experimental:
6
6
:page-noindex: true
7
7
8
-
NOTE: Chunk by CircleCI is currently in *open preview*. There are no extra costs during beta, Chunk uses CircleCI credits and your AI model provider token. Chunk tasks will be a paid feature after the beta.
8
+
NOTE: Chunk by CircleCI is currently in *beta*. There are no extra costs during beta, Chunk uses CircleCI credits and your AI model provider token. Chunk tasks will be a paid feature when generally available.
9
9
10
10
TIP: Have feedback or feature requests? Submit them on our link:https://circleci.canny.io/cloud-feature-requests?selectedCategory=chunk-ai[Ideas board, window=_blank] where you can also see existing feature requests and vote on them.
To improve verification success, create an "agent environment" CircleCI YAML file. Copy the environment setup parts of your existing CircleCI configuration into a dedicated file for Chunk.
75
-
76
-
* Name the file `cci-agent-setup.yml` and save it to your `.circleci` directory on your default branch.
77
-
* `cci-agent-setup.yml` needs to include a single workflow (the name of the workflow can be anything you want) with a single job named `cci-agent-setup`. The `cci-agent-setup` job needs to set up your environment for Chunk to use. You do not need to include any steps to run tests, this is purely for environment setup.
78
-
+
79
-
.Example config file for cci-agent-setup.yml
80
-
[source,yaml]
81
-
----
82
-
version: 2.1
83
-
workflows:
84
-
main:
85
-
jobs:
86
-
- cci-agent-setup
87
-
jobs:
88
-
cci-agent-setup:
89
-
docker:
90
-
- image: cimg/python:3.12
91
-
- image: cimg/postgres:15.3
92
-
steps:
93
-
- checkout
94
-
- run:
95
-
name: Hello World
96
-
command: |
97
-
echo "Hello, World!"
98
-
# insert more environment setup here
99
-
----
100
-
101
-
Chunk supports all standard CircleCI configuration options. This includes executors, resource classes, caching, contexts, environment variables, service containers, orbs, and everything else you would use in a standard CircleCI pipeline. If it works in your `.circleci/config.yml`, it works in `cci-agent-setup.yml`. For a complete reference of available configuration options, see the xref:reference:ROOT:configuration-reference.adoc[CircleCI Configuration Reference].
102
-
103
-
==== Example cci-agent-setup.yml files
104
-
105
-
[tabs]
106
-
====
107
-
Python::
108
-
+
109
-
--
110
-
[,yaml]
111
-
----
112
-
version: 2.1
113
-
workflows:
114
-
cci-agent-setup:
115
-
jobs:
116
-
- cci-agent-setup
117
-
jobs:
118
-
cci-agent-setup:
119
-
docker:
120
-
- image: cimg/python:3.12
121
-
- image: cimg/postgres:15.3
122
-
steps:
123
-
- checkout
124
-
- run:
125
-
name: Install dependencies
126
-
command: |
127
-
pip install -r requirements.txt
128
-
----
129
-
--
130
-
Caching & contexts::
131
-
+
132
-
--
133
-
[,yaml]
134
-
----
135
-
version: 2.1
136
-
workflows:
137
-
cci-agent-setup:
138
-
jobs:
139
-
- cci-agent-setup:
140
-
context:
141
-
- my-team-context # Includes any secrets/env vars from this context
Project environment variables:: Chunk automatically has access to any environment variables you have configured at the project level in CircleCI. You do not need to recreate or reference these, they are already available.
220
-
221
-
Contexts:: If you are using CircleCI contexts to manage secrets or environment variables, you must include the context in your `cci-agent-setup` job (as shown in the caching example above). Chunk will have access to all variables from that context, you do not need to manually recreate them.
222
-
223
-
==== Testing your environment setup
224
-
To build and iterate on Chunk's environment follow these steps:
225
-
226
-
. Navigate to menu:Organization Settings[Chunk Tasks]
227
-
. Identify your desired agent task.
228
-
. Select the ellipsis icon (image:guides:ROOT:icons/more.svg[ellipsis icon, role="no-border"]) and select btn:[Chunk Environment].
229
-
230
-
This page lets you run the contents of your cci-agent-setup.yml file on a specific branch and immediately see the results from those ad-hoc tasks. Use the btn:[Custom] button to submit a task to Chunk and see the results.
231
-
232
-
Merge the `cci-agent-setup.yml` file to your default branch when the results on the environment setup page are satisfactory.
233
-
234
-
==== Additional guidance for Chunk
235
-
To improve Chunk's ability to run tests and produce fixes that are aligned with stylistic/architectural preferences, you can include instructions. Your instructions can be in a markdown file (`claude.md` or `agents.md`) in the root of your repository or in a custom `cci-agent-setup.yml` file. Chunk should pick this up automatically.
74
+
To improve verification success, create an "agent environment" CircleCI YAML file to help Chunk set up the environment in which it runs your tests. See the xref:guides:toolkit:chunk-setup-and-overview.adoc#chunk-environment-setup[Chunk Environment Setup] section for more information.
236
75
237
76
== How Chunk's flaky test fix process works
238
77
@@ -309,16 +148,17 @@ Pull requests contain code diffs showing what changes Chunk recommends, along wi
309
148
310
149
== The Chunk tasks dashboard
311
150
312
-
Once you have set up some Chunk tasks, you can view an activity timeline on the Chunk tasks dashboard.
151
+
Once you have assigned Chunk some tasks, you can view an activity timeline on the Chunk dashboard. Tasks are available in a list view below the chat interface.
image::guides:ROOT:chunk/chunk-chat.png[Chunk tasks listed in chat view]
315
155
316
156
Once a fix is available the row is marked as "PR opened". Select a row to view the task overview. This includes the following information:
317
157
318
-
* Summary of the fix
319
-
* Root cause of the flakiness
320
-
* Details of the proposed fix
321
-
* Details of the level of verification achieved
158
+
* Summary of the fix.
159
+
* The root cause of the flakiness.
160
+
* The details of the proposed fix.
161
+
* The details of the level of verification achieved.
322
162
323
163
You also get a code diff of the proposed fix along with logs of the decision process presented as a conversation between "User" (Chunk) and "Assistant" (AI model provider). The diff and logs are designed to help you understand Chunk's reasoning and analysis process.
324
164
@@ -335,9 +175,9 @@ The following table shows the configuration options available when setting up Ch
335
175
336
176
|Run frequency
337
177
|How often Chunk analyzes and fixes flaky tests
338
-
a|* Daily (Sunday through Thursday at 22:00 UTC )
339
-
* Weekly every Sunday at 22:00 UTC (default)
340
-
* Monthly on the first day of the month at 22:00 UTC
178
+
a|* Daily (Sunday through Thursday at 22:00 UTC).
179
+
* Weekly every Sunday at 22:00 UTC (default).
180
+
* Monthly on the first day of the month at 22:00 UTC.
341
181
342
182
|Maximum tests to fix per run
343
183
|Limits the number of tests Chunk will attempt to fix in a single execution.
0 commit comments