|
1 | 1 | # CodeceptJS ESM Migration Plan |
2 | 2 |
|
3 | | -## Executive Summary |
| 3 | +This project is migrating from 3.x to ESM replacing all require() calls with imports. |
4 | 4 |
|
5 | | -Migrating CodeceptJS to ESM will require a phased approach due to: |
| 5 | +## Compare with Reference |
6 | 6 |
|
7 | | -- **900+ require() statements** across the codebase |
8 | | -- Complex dynamic module loading system in `container.js` |
9 | | -- Extensive plugin architecture with runtime module resolution |
10 | | -- Heavy use of conditional requires and try-catch patterns |
| 7 | +Each time unsure what to do refer to the same file implementation in **3.x branch** |
| 8 | +3.x branch contains stable version of all core classes that work |
11 | 9 |
|
12 | | -## Analysis Results |
| 10 | +## Running Tests |
13 | 11 |
|
14 | | -### Total Count of require() Statements |
| 12 | +Focus on acceptance tests: |
15 | 13 |
|
16 | | -- **850 require() statements** found in the `/lib` directory alone |
17 | | -- Additional requires in `/bin`, `/translations`, and root files |
18 | | -- **Total estimated across codebase: ~900+ require() statements** |
| 14 | +For instance this helps to understand if final migration is ok: |
19 | 15 |
|
20 | | -### Files with Most require() Statements |
21 | | - |
22 | | -1. `/lib/codecept.js` - 30 requires |
23 | | -2. `/lib/helper/Playwright.js` - 28 requires |
24 | | -3. `/lib/helper/Puppeteer.js` - 26 requires |
25 | | -4. `/lib/helper/WebDriver.js` - 24 requires |
26 | | -5. `/lib/helper/Protractor.js` - 22 requires |
27 | | -6. `/lib/workers.js` - 20 requires |
28 | | -7. `/lib/index.js` - 19 requires |
29 | | -8. `/lib/container.js` - 19 requires |
30 | | -9. `/lib/command/init.js` - 19 requires |
31 | | -10. `/lib/helper/TestCafe.js` - 17 requires |
32 | | - |
33 | | -### Problematic Patterns Requiring Special Handling |
34 | | - |
35 | | -#### 1. Dynamic require() Statements (Most Critical) |
36 | | - |
37 | | -- **Template-based requires**: `/lib/command/generate.js` line 158: `actor = require('${actorPath}')` |
38 | | -- **Conditional module loading**: `/lib/command/init.js` line 287: `require(\`../helper/\${helperName}\`)` |
39 | | -- **Plugin loading**: Multiple files use `require(module)` where module is a variable |
40 | | - |
41 | | -#### 2. Conditional require() Statements (High Impact) |
42 | | - |
43 | | -Found **75+ files** with conditional requires using `if` statements: |
44 | | - |
45 | | -- `/lib/container.js`: Module loading with fallback logic |
46 | | -- `/lib/config.js`: TypeScript support with `require('ts-node/register')` |
47 | | -- `/lib/codecept.js`: Global object loading based on configuration |
48 | | -- Helper files: Conditional loading of browser automation libraries |
49 | | - |
50 | | -#### 3. Try-catch require() Patterns (High Impact) |
51 | | - |
52 | | -Found **57+ files** with try-catch require patterns: |
53 | | - |
54 | | -- `/lib/utils.js`: `requireWithFallback()` function for graceful fallbacks |
55 | | -- `/lib/plugin/wdio.js`: `safeRequire()` function |
56 | | -- `/lib/helper/REST.js`: Dependency checking |
57 | | -- `/lib/container.js`: Module loading with error handling |
58 | | - |
59 | | -#### 4. require.resolve() Usage (Medium Impact) |
60 | | - |
61 | | -Found **5 files** using `require.resolve()`: |
62 | | - |
63 | | -- `/lib/utils.js`: Package existence checking in `requireWithFallback()` |
64 | | -- `/lib/helper/Puppeteer.js`: Module resolution |
65 | | -- `/lib/helper/extras/React.js`: Path resolution |
66 | | -- `/lib/helper/ApiDataFactory.js` and `/lib/helper/GraphQLDataFactory.js`: Dependency checking |
67 | | - |
68 | | -#### 5. \_\_dirname Usage (Medium Impact) |
69 | | - |
70 | | -Found **9 files** using `__dirname`: |
71 | | - |
72 | | -- `/lib/codecept.js`: Package.json path resolution |
73 | | -- `/lib/utils.js`: Local installation detection |
74 | | -- `/lib/workers.js`: Worker script path resolution |
75 | | -- `/lib/command/generate.js`: Template file paths |
76 | | -- `/lib/command/run-multiple.js`: Executable paths |
77 | | -- `/lib/helper/Nightmare.js`: Client script injection |
78 | | -- `/lib/helper/REST.js`: Certificate file paths (in documentation) |
79 | | -- `/lib/mocha/factory.js`: UI module paths |
80 | | -- `/lib/helper/testcafe/testcafe-utils.js`: Directory resolution |
81 | | - |
82 | | -## Migration Plan |
83 | | - |
84 | | -### Phase 1: Foundation (Estimated 2-3 weeks) |
85 | | - |
86 | | -#### 1.1 Update package.json |
87 | | - |
88 | | -```json |
89 | | -{ |
90 | | - "type": "module", |
91 | | - "exports": { |
92 | | - ".": "./lib/index.js", |
93 | | - "./lib/*": "./lib/*.js" |
94 | | - }, |
95 | | - "engines": { |
96 | | - "node": ">=14.13.1" |
97 | | - } |
98 | | -} |
99 | 16 | ``` |
| 17 | +DEBUG="codeceptjs:*" ./bin/codecept.js run --config test/acceptance/codecept.Playwright.js --verbose |
100 | 18 |
|
101 | | -#### 1.2 Create ESM compatibility layer |
102 | | - |
103 | | -- Create `lib/compat/moduleLoader.js` for dynamic imports |
104 | | -- Convert `__dirname`/`__filename` to `import.meta.url` |
105 | | -- Replace `require.resolve()` with `import.meta.resolve()` |
106 | | - |
107 | | -#### 1.3 Convert bin/ entry points |
108 | | - |
109 | | -- `bin/codecept.js` → ESM imports |
110 | | -- Update shebang and command structure |
111 | | - |
112 | | -### Phase 2: Core Infrastructure (Estimated 3-4 weeks) |
113 | | - |
114 | | -#### 2.1 Convert container.js (CRITICAL) |
115 | | - |
116 | | -Key changes needed in `lib/container.js:285-305`: |
117 | | - |
118 | | -```javascript |
119 | | -// Current problematic code: |
120 | | -const mod = require(moduleName) |
121 | | -HelperClass = mod.default || mod |
122 | | - |
123 | | -// ESM replacement: |
124 | | -const mod = await import(moduleName) |
125 | | -HelperClass = mod.default || mod |
| 19 | +DEBUG="codeceptjs:*" ./bin/codecept.js run --config test/acceptance/codecept.Playwright.js --debug --grep within |
126 | 20 | ``` |
127 | 21 |
|
128 | | -#### 2.2 Update dynamic loading functions |
129 | | - |
130 | | -- `requireHelperFromModule()` → `importHelperFromModule()` |
131 | | -- `loadSupportObject()` → async with dynamic imports |
132 | | -- `createPlugins()` → async plugin loading |
133 | | - |
134 | | -#### 2.3 Convert core files |
135 | | - |
136 | | -Priority order: |
137 | | - |
138 | | -1. `lib/utils.js` (19 requires) |
139 | | -2. `lib/index.js` (19 requires) |
140 | | -3. `lib/codecept.js` (30 requires) |
141 | | -4. `lib/config.js` (13 requires) |
142 | | - |
143 | | -### Phase 3: Helper System (Estimated 4-5 weeks) |
144 | | - |
145 | | -#### 3.1 Convert browser automation helpers |
146 | | - |
147 | | -High-impact files: |
| 22 | +Do not say: it works befire running specific acceptance test. |
| 23 | +Tests may stuck so always run them with timeout call: |
148 | 24 |
|
149 | | -- `lib/helper/Playwright.js` (28 requires) |
150 | | -- `lib/helper/Puppeteer.js` (26 requires) |
151 | | -- `lib/helper/WebDriver.js` (24 requires) |
152 | | -- `lib/helper/TestCafe.js` (17 requires) |
153 | | - |
154 | | -#### 3.2 Handle conditional dependencies |
155 | | - |
156 | | -Convert try-catch patterns: |
157 | | - |
158 | | -```javascript |
159 | | -// Current: |
160 | | -try { |
161 | | - const puppeteer = require('puppeteer') |
162 | | -} catch (e) { |
163 | | - // fallback |
164 | | -} |
165 | | - |
166 | | -// ESM: |
167 | | -let puppeteer |
168 | | -try { |
169 | | - puppeteer = await import('puppeteer') |
170 | | -} catch (e) { |
171 | | - // fallback |
172 | | -} |
| 25 | +``` |
| 26 | +timeout=30000 ./bin/codecept.js run --config test/acceptance/codecept.Playwright.js --verbose |
173 | 27 | ``` |
174 | 28 |
|
175 | | -### Phase 4: Commands & Plugins (Estimated 2-3 weeks) |
176 | | - |
177 | | -#### 4.1 Convert command files |
178 | | - |
179 | | -- `lib/command/init.js` (19 requires) |
180 | | -- `lib/command/generate.js` (template loading) |
181 | | -- `lib/command/run*.js` files |
182 | | - |
183 | | -#### 4.2 Update plugin system |
184 | | - |
185 | | -- Support both CJS and ESM plugins |
186 | | -- Async plugin initialization |
187 | | -- Plugin discovery mechanism |
188 | | - |
189 | | -### Phase 5: Testing & Validation (Estimated 2-3 weeks) |
190 | | - |
191 | | -#### 5.1 Test compatibility |
192 | | - |
193 | | -- All existing tests must pass |
194 | | -- Plugin ecosystem compatibility |
195 | | -- Helper loading in various configurations |
196 | | - |
197 | | -#### 5.2 Documentation updates |
198 | | - |
199 | | -- Migration guide for users |
200 | | -- Plugin development guidelines |
201 | | -- Breaking changes documentation |
202 | | - |
203 | | -## Critical Migration Challenges |
204 | | - |
205 | | -### 1. Dynamic Module Loading |
206 | | - |
207 | | -The biggest challenge is in `container.js` where modules are loaded dynamically based on configuration. This requires converting synchronous `require()` to asynchronous `import()`. |
208 | | - |
209 | | -**Solution**: Make container creation async and update all callers. |
210 | | - |
211 | | -### 2. Plugin Ecosystem |
212 | | - |
213 | | -Many plugins may still use CommonJS. |
214 | | - |
215 | | -**Solution**: Support both formats during transition period. |
216 | | - |
217 | | -### 3. Global Object Injection |
218 | | - |
219 | | -`codecept.js` conditionally adds globals based on config. |
220 | | - |
221 | | -**Solution**: Maintain compatibility layer for existing configurations. |
222 | | - |
223 | | -### 4. File Path Resolution |
224 | | - |
225 | | -Extensive use of `__dirname` for path resolution. |
226 | | - |
227 | | -**Solution**: Create utility functions using `import.meta.url`. |
228 | | - |
229 | | -## Compatibility Strategy |
230 | | - |
231 | | -### Dual Package Approach (Recommended) |
232 | | - |
233 | | -1. Maintain CommonJS build for compatibility |
234 | | -2. Provide ESM build for modern usage |
235 | | -3. Use `package.json` exports field to serve appropriate version |
236 | | - |
237 | | -### Breaking Changes |
238 | | - |
239 | | -- Drop Node.js < 14.13.1 support |
240 | | -- Async container initialization |
241 | | -- Some plugin APIs may need updates |
242 | | - |
243 | | -## Timeline Summary |
244 | | - |
245 | | -- **Total estimated time**: 13-18 weeks |
246 | | -- **Critical path**: Container.js and dynamic loading |
247 | | -- **Risk areas**: Plugin compatibility, helper loading |
248 | | - |
249 | | -## Next Steps |
| 29 | +Web Server for this tests are running on port 8000 and it works but responds 500 for HEAD requests. |
250 | 30 |
|
251 | | -1. Start with Phase 1 foundation work |
252 | | -2. Create comprehensive test suite for migration validation |
253 | | -3. Engage with plugin maintainers early |
254 | | -4. Consider feature freeze during migration |
255 | | -5. Plan gradual rollout strategy |
| 31 | +## Princinples |
256 | 32 |
|
257 | | -## Testing Commands |
| 33 | +All unit tests for Playwright/Puppeteer/WebdriverIO are passing. But acceptance tests are failing due to promises composition. Complexity of promises composition comes from promise chaining and async/await syntax. And `session` mechanism from `promise.js` which allows spawning different promises chains and sync them up in the end. |
258 | 34 |
|
259 | | -To test the project: |
| 35 | +The full test suite extensively uses plugins so refer to plugins and corresponding helpers as well |
260 | 36 |
|
261 | | -- `npm run lint` - Run linting |
262 | | -- `npm run test:unit` - Run unit tests |
263 | | -- `npm run test:runner` - Run runner tests |
264 | | -- `npm run test` - Run both unit and runner tests |
| 37 | +We are building test framework. So even passing ests are not indicator of successful usage. Side effects like: timeouts, bad output, errors, memory leaks, etc are affecting performance. |
| 38 | +For instance, if a tests stuck and won't finish this is very bad. |
265 | 39 |
|
266 | | -## Current Migration Status |
| 40 | +## Coding Style |
267 | 41 |
|
268 | | -Starting with creating example-esm project for iterative testing and validation. |
| 42 | +- Never add comments unless explicitly required. |
| 43 | +- DO not mention plugins or helpers inside core classes |
0 commit comments