You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/guides/modules/test/pages/smarter-testing.adoc
+18-15Lines changed: 18 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
= Smarter Testing
2
2
:page-badge: Preview
3
3
:page-platform: Cloud
4
-
:page-description: This document describes the Smarter Testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes.
4
+
:page-description: This page describes CircleCI's Smarter Testing. Only run tests that are impacted by code changes and evenly distribute tests across parallel execution nodes.
5
5
:experimental:
6
6
:page-noindex: true
7
7
:page-aliases: adaptive-testing.adoc
@@ -43,8 +43,8 @@ TIP: In codebases with sparse test coverage, Smarter Testing cannot accurately d
43
43
== How it works
44
44
Smarter Testing operates through two main components that work together to optimize your test execution:
45
45
46
-
* Dynamic test splitting
47
-
* Test impact analysis
46
+
* Dynamic test splitting.
47
+
* Test impact analysis.
48
48
49
49
Each component is described in more detail below.
50
50
@@ -551,7 +551,10 @@ a| * `all` selects and runs all discovered tests, used to run the full test suit
551
551
552
552
== 3. Start using Smarter Testing
553
553
554
-
Now the test suite is set up, test selection is working and the test analysis is up to date with the latest changes from the feature branch that ran the first test analysis.
554
+
Now the test suite is set up:
555
+
556
+
* Test selection is working.
557
+
* The test analysis is up to date with the latest changes from the feature branch that ran the first test analysis.
555
558
556
559
*Action Items*
557
560
@@ -679,9 +682,9 @@ The Smarter Testing feature has some limitations to consider:
679
682
680
683
*Debugging steps:*
681
684
682
-
. Check that all test files are discovered with the discover command
683
-
. Verify parallelism is set correctly in your config.yml
684
-
. Look for timing data in previous test runs
685
+
. Check that all test files are discovered with the discover command.
686
+
. Verify parallelism is set correctly in your config.yml.
687
+
. Look for timing data in previous test runs.
685
688
. Ensure test results are being stored with `store_test_results`.
686
689
687
690
=== Test impact analysis not selecting expected tests
@@ -692,10 +695,10 @@ The Smarter Testing feature has some limitations to consider:
692
695
693
696
*Debugging steps:*
694
697
695
-
. Verify analysis has run successfully on your configured branch(es)
696
-
. Check that coverage data is being generated correctly
697
-
. Review the full-test-run-paths configuration - changes to these paths trigger full test runs
698
-
. Confirm the analysis command is producing valid LCOV output
698
+
. Verify analysis has run successfully on your configured branch(es).
699
+
. Check that coverage data is being generated correctly.
700
+
. Review the full-test-run-paths configuration - changes to these paths trigger full test runs.
701
+
. Confirm the analysis command is producing valid LCOV output.
699
702
700
703
*When all tests run:* If no impact data exists or all tests are determined to be affected, the system runs all tests as a safety measure.
701
704
@@ -745,9 +748,9 @@ The frequency depends on your test execution speed and development pace:
745
748
746
749
*Consider re-running analysis:*
747
750
748
-
* After major refactoring or code restructuring
749
-
* When test selection seems inaccurate or outdated
750
-
* After adding significant new code or tests
751
+
* After major refactoring or code restructuring.
752
+
* When test selection seems inaccurate or outdated.
753
+
* After adding significant new code or tests.
751
754
752
755
*Remember:* You can customize which branches run analysis through your CircleCI configuration - it does not have to be limited to the main branch.
753
756
@@ -822,7 +825,7 @@ See the <<run-higher-parallelism-on-the-analysis-branch,Run higher parallelism o
822
825
[#baseline-coverage]
823
826
=== Why are there so many files impacting a test?
824
827
825
-
If you see many files impacting each test during analysis (for example, "...found 150 files impacting test..."), this may be caused by shared setup code like global imports or framework initialization being included in coverage.
828
+
If you see many files impacting each test during analysis, for example, `...found 150 files impacting test...`, this may be caused by shared setup code like global imports or framework initialization being included in coverage.
826
829
827
830
This extraneous coverage can be excluded by providing an `analysis-baseline` command to compute the code covered during startup that isn't directly exercised by test code. We call this "baseline coverage data".
0 commit comments