Skip to content

Commit f23f9c4

Browse files
committed
move environment setup section
1 parent 1b45ba3 commit f23f9c4

File tree

5 files changed

+193
-173
lines changed

5 files changed

+193
-173
lines changed
-56.2 KB
Loading
158 KB
Loading
-132 KB
Binary file not shown.

docs/guides/modules/test/pages/fix-flaky-tests.adoc

Lines changed: 13 additions & 173 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
= Fix flaky tests
2-
:page-badge: Preview
2+
:page-badge: Beta
33
:page-platform: Cloud
44
:page-description: Learn about Chunk by CircleCI and how it can automatically identify and fix flaky tests in your CI/CD pipelines.
55
:experimental:
66
:page-noindex: true
77

8-
NOTE: Chunk by CircleCI is currently in *open preview*. There are no extra costs during beta, Chunk uses CircleCI credits and your AI model provider token. Chunk tasks will be a paid feature after the beta.
8+
NOTE: Chunk by CircleCI is currently in *beta*. There are no extra costs during beta, Chunk uses CircleCI credits and your AI model provider token. Chunk tasks will be a paid feature when generally available.
99

1010
TIP: Have feedback or feature requests? Submit them on our link:https://circleci.canny.io/cloud-feature-requests?selectedCategory=chunk-ai[Ideas board, window=_blank] where you can also see existing feature requests and vote on them.
1111

@@ -71,168 +71,7 @@ image::guides:ROOT:chunk/assign-task.png[Chunk assign task modal]
7171

7272
=== Chunk environment setup
7373

74-
To improve verification success, create an "agent environment" CircleCI YAML file. Copy the environment setup parts of your existing CircleCI configuration into a dedicated file for Chunk.
75-
76-
* Name the file `cci-agent-setup.yml` and save it to your `.circleci` directory on your default branch.
77-
* `cci-agent-setup.yml` needs to include a single workflow (the name of the workflow can be anything you want) with a single job named `cci-agent-setup`. The `cci-agent-setup` job needs to set up your environment for Chunk to use. You do not need to include any steps to run tests, this is purely for environment setup.
78-
+
79-
.Example config file for cci-agent-setup.yml
80-
[source,yaml]
81-
----
82-
version: 2.1
83-
workflows:
84-
main:
85-
jobs:
86-
- cci-agent-setup
87-
jobs:
88-
cci-agent-setup:
89-
docker:
90-
- image: cimg/python:3.12
91-
- image: cimg/postgres:15.3
92-
steps:
93-
- checkout
94-
- run:
95-
name: Hello World
96-
command: |
97-
echo "Hello, World!"
98-
# insert more environment setup here
99-
----
100-
101-
Chunk supports all standard CircleCI configuration options. This includes executors, resource classes, caching, contexts, environment variables, service containers, orbs, and everything else you would use in a standard CircleCI pipeline. If it works in your `.circleci/config.yml`, it works in `cci-agent-setup.yml`. For a complete reference of available configuration options, see the xref:reference:ROOT:configuration-reference.adoc[CircleCI Configuration Reference].
102-
103-
==== Example cci-agent-setup.yml files
104-
105-
[tabs]
106-
====
107-
Python::
108-
+
109-
--
110-
[,yaml]
111-
----
112-
version: 2.1
113-
workflows:
114-
cci-agent-setup:
115-
jobs:
116-
- cci-agent-setup
117-
jobs:
118-
cci-agent-setup:
119-
docker:
120-
- image: cimg/python:3.12
121-
- image: cimg/postgres:15.3
122-
steps:
123-
- checkout
124-
- run:
125-
name: Install dependencies
126-
command: |
127-
pip install -r requirements.txt
128-
----
129-
--
130-
Caching & contexts::
131-
+
132-
--
133-
[,yaml]
134-
----
135-
version: 2.1
136-
workflows:
137-
cci-agent-setup:
138-
jobs:
139-
- cci-agent-setup:
140-
context:
141-
- my-team-context # Includes any secrets/env vars from this context
142-
jobs:
143-
cci-agent-setup:
144-
docker:
145-
- image: cimg/node:18.0
146-
steps:
147-
- checkout
148-
- restore_cache:
149-
keys:
150-
- v1-dependencies-{{ checksum "package-lock.json" }}
151-
- run:
152-
name: Install dependencies
153-
command: npm install
154-
- save_cache:
155-
paths:
156-
- node_modules
157-
key: v1-dependencies-{{ checksum "package-lock.json" }}
158-
----
159-
--
160-
Multiple services::
161-
+
162-
--
163-
[,yaml]
164-
----
165-
version: 2.1
166-
workflows:
167-
cci-agent-setup:
168-
jobs:
169-
- cci-agent-setup
170-
jobs:
171-
cci-agent-setup:
172-
docker:
173-
- image: cimg/ruby:3.2
174-
- image: cimg/postgres:15.3
175-
environment:
176-
POSTGRES_USER: circleci
177-
POSTGRES_DB: test_db
178-
- image: redis:7.0
179-
steps:
180-
- checkout
181-
- run:
182-
name: Wait for DB
183-
command: dockerize -wait tcp://localhost:5432 -timeout 1m
184-
- run:
185-
name: Install dependencies
186-
command: bundle install
187-
- run:
188-
name: Setup database
189-
command: bundle exec rake db:setup
190-
----
191-
--
192-
Resource classes & machine::
193-
+
194-
--
195-
[,yaml]
196-
----
197-
version: 2.1
198-
workflows:
199-
cci-agent-setup:
200-
jobs:
201-
- cci-agent-setup
202-
jobs:
203-
cci-agent-setup:
204-
machine:
205-
image: ubuntu-2204:2024.01.2
206-
resource_class: large
207-
steps:
208-
- checkout
209-
- run:
210-
name: Install dependencies
211-
command: |
212-
sudo apt-get update
213-
sudo apt-get install -y build-essential
214-
----
215-
--
216-
====
217-
218-
==== Environment variables and contexts
219-
Project environment variables:: Chunk automatically has access to any environment variables you have configured at the project level in CircleCI. You do not need to recreate or reference these, they are already available.
220-
221-
Contexts:: If you are using CircleCI contexts to manage secrets or environment variables, you must include the context in your `cci-agent-setup` job (as shown in the caching example above). Chunk will have access to all variables from that context, you do not need to manually recreate them.
222-
223-
==== Testing your environment setup
224-
To build and iterate on Chunk's environment follow these steps:
225-
226-
. Navigate to menu:Organization Settings[Chunk Tasks]
227-
. Identify your desired agent task.
228-
. Select the ellipsis icon (image:guides:ROOT:icons/more.svg[ellipsis icon, role="no-border"]) and select btn:[Chunk Environment].
229-
230-
This page lets you run the contents of your cci-agent-setup.yml file on a specific branch and immediately see the results from those ad-hoc tasks. Use the btn:[Custom] button to submit a task to Chunk and see the results.
231-
232-
Merge the `cci-agent-setup.yml` file to your default branch when the results on the environment setup page are satisfactory.
233-
234-
==== Additional guidance for Chunk
235-
To improve Chunk's ability to run tests and produce fixes that are aligned with stylistic/architectural preferences, you can include instructions. Your instructions can be in a markdown file (`claude.md` or `agents.md`) in the root of your repository or in a custom `cci-agent-setup.yml` file. Chunk should pick this up automatically.
74+
To improve verification success, create an "agent environment" CircleCI YAML file to help Chunk set up the environment in which it runs your tests. See the xref:guides:toolkit:chunk-setup-and-overview.adoc#chunk-environment-setup[Chunk Environment Setup] section for more information.
23675

23776
== How Chunk's flaky test fix process works
23877

@@ -309,16 +148,17 @@ Pull requests contain code diffs showing what changes Chunk recommends, along wi
309148

310149
== The Chunk tasks dashboard
311150

312-
Once you have set up some Chunk tasks, you can view an activity timeline on the Chunk tasks dashboard.
151+
Once you have assigned Chunk some tasks, you can view an activity timeline on the Chunk dashboard. Tasks are available in a list view below the chat interface.
313152

314-
image::guides:ROOT:chunk/chunk-tasks-dashboard.png[Chunk tasks dashboard]
153+
.See Chunk tasks listed below chat interface
154+
image::guides:ROOT:chunk/chunk-chat.png[Chunk tasks listed in chat view]
315155

316156
Once a fix is available the row is marked as "PR opened". Select a row to view the task overview. This includes the following information:
317157

318-
* Summary of the fix
319-
* Root cause of the flakiness
320-
* Details of the proposed fix
321-
* Details of the level of verification achieved
158+
* Summary of the fix.
159+
* The root cause of the flakiness.
160+
* The details of the proposed fix.
161+
* The details of the level of verification achieved.
322162

323163
You also get a code diff of the proposed fix along with logs of the decision process presented as a conversation between "User" (Chunk) and "Assistant" (AI model provider). The diff and logs are designed to help you understand Chunk's reasoning and analysis process.
324164

@@ -335,9 +175,9 @@ The following table shows the configuration options available when setting up Ch
335175

336176
|Run frequency
337177
|How often Chunk analyzes and fixes flaky tests
338-
a|* Daily (Sunday through Thursday at 22:00 UTC )
339-
* Weekly every Sunday at 22:00 UTC (default)
340-
* Monthly on the first day of the month at 22:00 UTC
178+
a|* Daily (Sunday through Thursday at 22:00 UTC).
179+
* Weekly every Sunday at 22:00 UTC (default).
180+
* Monthly on the first day of the month at 22:00 UTC.
341181

342182
|Maximum tests to fix per run
343183
|Limits the number of tests Chunk will attempt to fix in a single execution.

0 commit comments

Comments
 (0)