Skip to content

Commit de919ff

Browse files
Update copilot-library.md with new sections and details...
...including how to give up, common commands, session finishing tasks, debugging tips, compiler tests, and fourslash testing.
1 parent 47b5b29 commit de919ff

File tree

1 file changed

+32
-4
lines changed

1 file changed

+32
-4
lines changed

.github/copilot-library.md

Lines changed: 32 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,10 @@ When you think you're done, there are some mandatory things you need to do befor
2828

2929
If you have a fix, explain in 1-4 paragraphs your understanding of the origin of the bug and why your fix is the correct one.
3030

31-
If you haven't already, run `hereby runtests-parallel` and ensure there are zero errors. Read the sections on Baseline Tests if you don't know how to manage the output of baseline diffs.
31+
If you haven't already, run `hereby runtests-parallel` and ensure there are zero errors.
32+
You should expect a full test run to take up to 20 minutes to finish.
33+
Read the sections on Baseline Tests if you don't know how to manage the output of baseline diffs.
34+
Remember, there is no such thing as an "unrelated" failure in this repo!
3235

3336
Next, run `hereby lint`. Fix any errors reported here. Lint errors will generally never cause new test failures to appear.
3437

@@ -60,13 +63,29 @@ If you run a specific test using `hereby runtests -t testName`, you will see the
6063

6164
# Compiler Tests
6265

66+
> How to write, run, and manage tests related to core tsc functionality (scan, parse, bind, check, and emit)
67+
68+
In general, all testcases you add related to core checker behavior should be in the form of baseline tests.
69+
These tests validate TypeScript behavior, type checking, symbol resolution, and error reporting.
70+
You should read the "Dealing with Baselines" topic once you've written a test.
71+
72+
## Dealing with Baselines
73+
6374
> How to work with baseline-based tests like those in `tests/cases/compiler`
6475
65-
In general, all testcases you add related to core checker behavior should be in the form of baseline tests. These tests validate TypeScript behavior, type checking, symbol resolution, and error reporting. When these tests run, they either pass (because the output matches the reference copy in `tests/baselines/reference`), or fail and create a new file in `tests/baselines/local`.
76+
When compiler tests run, they create baseline files.
77+
If these baseline files match what's already in the repo, the test passes.
78+
Otherwise, the test fails, and a new file appears in `tests/baselines/local`.
79+
80+
The failure when you create a new test is expected; for new content, example the baseline output to see that it matches what you expect, and run `hereby baseline-accept`.
81+
82+
Often, a correct bufix will still cause baseline differences.
83+
You need to analyze the difference between the new baseline in `tests/baselines/local` and the checked-in version at `tests/baselines/reference`, and determine for yourself if the change is desirable or not.
6684

67-
The failure when you create a new test is expected; for new content, example the baseline output to see that it matches what you expect, and run `hereby baseline-accept` to copy the new file to `tests/baselines/reference`
85+
If the change isn't desirable, adjust your bugfix and run the tests again. Iterate as needed.
6886

69-
When this failure occurs in other situations, compare the new file to the old file and make sure it's expected, and run `hereby baseline-accept`. This will generate a diff you should commit as part of your changes.
87+
If the change is desirable, run `hereby baseline-accept`, which will copy the new file to `tests/baselines/reference`.
88+
This will be a diff that you should submit as part of your PR.
7089

7190
## Creating
7291

@@ -87,6 +106,10 @@ The file format looks like this
87106
let x: string = 42; // Error expected
88107
```
89108
You can set any TypeScript compiler option using the `// @flag: value` syntax.
109+
If it's useful to test multiple flag values at once, you can use commas:
110+
```ts
111+
// @target: ES2015,ES2022
112+
```
90113

91114
The default file extension for the interior file is `.ts`, but you can change that (or create multiple files) with filename directives:
92115
```ts
@@ -114,6 +137,11 @@ class Derived extends Base {
114137
new Base(); // Should error - cannot instantiate abstract class
115138
```
116139

140+
You can run a test by name by running
141+
```
142+
npx hereby runtests -t filenameOfThatTest
143+
```
144+
where filename is just any substring, e.g. if you write `tests/cases/compiler/foo.ts`, run `hereby runtests -t foo`
117145

118146
# Fourslash Testing
119147

0 commit comments

Comments
 (0)