Skip to content

Commit 0cd9834

Browse files
committed
Update baseline docs for baseline command
1 parent 81ade3c commit 0cd9834

File tree

1 file changed

+23
-13
lines changed

1 file changed

+23
-13
lines changed

docs/guides/modules/test/pages/adaptive-testing.adoc

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -322,7 +322,6 @@ Supported coverage template variables:
322322

323323
* `<< outputs.lcov >>`: Coverage data in LCOV format.
324324
* `<< outputs.go-coverage >>`: Coverage data in Go coverage format.
325-
* `<< outputs.gcov >>`: Coverage data in `gcov` coverage format.
326325

327326
The coverage location does not need to be set in the outputs map, a temporary file will be created and used during analysis with the template variable from the analysis command.
328327

@@ -506,12 +505,6 @@ Any remaining tests will be analysed the next time test analysis is run.
506505
|true
507506
|Whether the tests should be distributed across a shared queue and fetched across multiple dynamic batches. +
508507
If a test runner has slow start up time per batch, disabling this can speed up tests.
509-
510-
| `coverage-baseline`
511-
|.circleci/example-baseline.info
512-
|Path to a baseline coverage file to subtract from test coverage during analysis. +
513-
Use this to exclude shared setup or initialization code from test impact data. +
514-
The baseline file can be in any supported coverage format. While it does not need to match your test coverage output format, using the same format (for example, LCOV format for `<< outputs.lcov >>`) is recommended for consistency. For more information, refer to the <<baseline-coverage>> section.
515508
|===
516509
--
517510

@@ -811,21 +804,38 @@ See Scenario 3 in the "Flag Usage Scenarios" section for examples of customizing
811804

812805
If you see many files impacting each test during analysis (for example, "...found 150 files impacting test..."), this may be caused by shared setup code like global imports or framework initialization being included in coverage.
813806

814-
*Solution: Use a coverage baseline to exclude setup code*
807+
This extraneous coverage can be excluded by providing an `analysis-baseline` command to compute the code covered during startup that isn't directly exercised by test code. We call this "baseline coverage data".
808+
809+
The `analysis-baseline` command must produce coverage output written a coverage template variable. The baseline coverage data can be in any supported coverage format. While it does not need to match your test coverage output format, using the same format (for example, LCOV format for `<< outputs.lcov >>`) is recommended for consistency.
815810

816-
. Create a minimal test that only does imports/setup (no test logic)
817-
. Generate coverage from that test and save it to `.circleci/example-baseline.info` (you can name the file however you'd like)
818-
. Reference it in your test suite:
811+
. Create a minimal test that only does imports/setup (no test logic), in the example below this is called `src/baseline/noop.test.ts`.
812+
. Add an `analysis-baseline` command to your test suite. This command will be broadly similar to your `analysis` command, except that it should only run the minimal test.
819813

820814
[source,yaml]
821815
----
822816
# .circleci/test-suites.yml
817+
name: ci tests
818+
discover: jest --listTests --testPathPattern=src/
819+
run: JEST_JUNIT_OUTPUT_FILE="<< outputs.junit >>" jest --runInBand --reporters=jest-junit --bail << test.atoms >>
820+
analysis: |
821+
jest --runInBand --silent --bail --coverage --coverageProvider=v8 \
822+
--coverage-directory="$(dirname << outputs.lcov >>)" \
823+
--coverageReporters=lcovonly \
824+
<< test.atoms >> \
825+
&& cat "$(dirname << outputs.lcov >>)"/*.info > << outputs.lcov >>
826+
analysis-baseline: |
827+
jest --runInBand --silent --bail --coverage --coverageProvider=v8 \
828+
--coverageReporters=lcovonly \
829+
--coverage-directory="$(dirname << outputs.lcov >>)" \
830+
"src/baseline/noop.test.ts" \
831+
&& cat "$(dirname << outputs.lcov >>)"/*.info > << outputs.lcov >>
832+
outputs:
833+
junit: test-reports/tests.xml
823834
options:
824835
adaptive-testing: true
825-
coverage-baseline: .circleci/example-baseline.info
826836
----
827837

828-
The coverage data in the baseline file will be subtracted from each test's coverage during analysis. Rerun analysis and you should see fewer impacting files per test. Note that the baseline file should be in the same format as your test coverage output (e.g., LCOV format for << outputs.lcov >>).
838+
The `analysis-baseline` command will be run just before running analysis. The coverage data produced by the `analysis-baseline` command will be subtracted from each test's coverage during analysis. Rerun analysis and you should see fewer impacting files per test.
829839

830840
=== What test frameworks are supported?
831841

0 commit comments

Comments
 (0)