Skip to content

Commit 7047e6b

Browse files
pditommasoclaude
pditommaso
and
claude
authored
Add Nextflow Development Constitution (#6578) [ci skip]
* Add Nextflow Development Constitution Introduce comprehensive development constitution documenting core principles and practices for Nextflow development including modular architecture, test-driven quality assurance, dataflow programming model, licensing compliance, DCO requirements, semantic versioning, and Groovy code standards. The constitution codifies existing best practices from CLAUDE.md and CONTRIBUTING.md to provide clear governance and quality standards for the project. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * wip Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * Update constitution [ci fast] Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> --------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
1 parent 20f4631 commit 7047e6b

File tree

1 file changed

+183
-35
lines changed

1 file changed

+183
-35
lines changed

‎.specify/memory/constitution.md‎

Lines changed: 183 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,198 @@
1-
# [PROJECT_NAME] Constitution
2-
<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
1+
# Nextflow Development Constitution
2+
3+
<!--
4+
SYNC IMPACT REPORT
5+
==================
6+
Version Change: INITIAL → 1.0.0 (Initial constitution)
7+
Modified Principles: N/A (new constitution)
8+
Added Sections:
9+
- All principles (I-VII)
10+
- Development Workflow
11+
- Quality Standards
12+
- Governance
13+
14+
Removed Sections: N/A (initial version)
15+
16+
Templates Status:
17+
✅ plan-template.md - Reviewed, aligned with modular architecture and testing principles
18+
✅ spec-template.md - Reviewed, aligned with user scenario focus and requirements structure
19+
✅ tasks-template.md - Reviewed, aligned with test-driven and parallel development principles
20+
✅ agent-file-template.md - Reviewed, no agent-specific conflicts
21+
✅ checklist-template.md - Reviewed, compatible with quality standards
22+
23+
Follow-up TODOs:
24+
- None at this time
25+
==================
26+
-->
327

428
## Core Principles
529

6-
### [PRINCIPLE_1_NAME]
7-
<!-- Example: I. Library-First -->
8-
[PRINCIPLE_1_DESCRIPTION]
9-
<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
30+
### I. Modular Architecture
31+
32+
Nextflow MUST maintain a clear separation between core functionality and extensions through its modular architecture:
33+
34+
- **Core modules** (`modules/`) contain essential functionality: core workflow runtime & plugin system (nextflow), shared utilities (nf-commons), language parsing (nf-lang), HTTP filesystem support (nf-httpfs), and lineage tracking (nf-lineage)
35+
- **Core plugins** (`plugins/`) provides cloud provider integrations (AWS, Azure, GCP), execution platforms (Kubernetes), and specialized services (Seqera Platform, Wave container management)
36+
- New features MUST be evaluated for placement: core features belong in `modules/`, specialized/cloud-specific features belong in `plugins/`
37+
- Each module and plugin MUST be independently buildable and testable
38+
- Plugin dependencies MUST be explicitly declared in `build.gradle` with semantic versioning
39+
40+
**Rationale**: This architecture enables independent development of cloud provider features without core engine changes, supports third-party plugin development, and maintains a clean separation of concerns across a large multi-module codebase.
41+
42+
### II. Test-Driven Quality Assurance (NON-NEGOTIABLE)
43+
44+
Testing MUST be comprehensive and multi-layered before any code is merged:
45+
46+
- **Unit tests** MUST use Spock Framework for all Groovy code, be independently executable, and achieve meaningful coverage (measured via JaCoCo)
47+
- **Integration tests** (`tests/` directory) MUST validate end-to-end workflows using actual `.nf` scripts with expected outputs
48+
- **Smoke tests** (`make smoke` or `NXF_SMOKE=1`) MUST be available to skip long-running and cloud-dependent tests during rapid development
49+
- **Cloud validation tests** (`validation/` directory) MUST verify cloud provider integrations end-to-end before release
50+
- **Documentation tests** (`docs/snippets/`) MUST ensure all documentation examples remain functional
51+
- All tests MUST pass before commits, and `make test` MUST be run locally before pushing
52+
53+
**Rationale**: Scientific workflows demand reliability and reproducibility. Multi-layered testing catches issues at appropriate levels: unit tests for logic, integration tests for workflow correctness, and validation tests for cloud provider compatibility.
54+
55+
### III. Dataflow Programming Model
56+
57+
Nextflow's core abstraction MUST adhere to following principles and guidelines:
58+
59+
- The dataflow programming model is the fundamental abstraction in the Nextflow programming model.
60+
- Workflows are defined as dataflow graphs where data flows between processes.
61+
- Processes MUST be stateless, side-effect-free transformations that communicate via channels.
62+
- Workflows should be defined in a platform agnostic manner to enable portability across clusters and clouds.
63+
- The DSL MUST prioritize expressiveness for concurrent and parallel pipeline definition.
64+
- Changes to the language parser (ANTLR grammars in `nf-lang`) MUST preserve backward compatibility with existing pipelines unless explicitly versioned (DSL1 vs DSL2).
65+
- Concurrency primitives (GPars actors/dataflow) MUST be used correctly to maintain the dataflow semantics.
66+
67+
**Rationale**: The dataflow model is Nextflow's fundamental abstraction, enabling automatic parallelization and distribution. Preserving this model ensures existing scientific pipelines continue to work and users can reason about workflow behavior.
68+
69+
### IV. Apache 2.0 License Compliance
70+
71+
All source code MUST include Apache 2.0 license headers:
72+
73+
- Every source file MUST begin with the Apache 2.0 license header
74+
- All contributions MUST comply with Apache 2.0 terms
75+
- Third-party dependencies MUST use compatible licenses
76+
- License compliance MUST be verified during code review
77+
78+
**Rationale**: Legal clarity protects both contributors and users. Consistent licensing enables academic and commercial use, which is critical for scientific software adoption.
79+
80+
### V. Developer Certificate of Origin (DCO) Sign-off
81+
82+
All commits MUST be signed with DCO certification:
83+
84+
- Contributors MUST certify they have the right to submit the code by using `git commit -s` or `git commit --signoff`
85+
- Every commit message MUST include a `Signed-off-by` line
86+
- The DCO bot MUST verify sign-off before any PR can be merged
87+
- Contributors MUST NOT bypass the DCO requirement
1088

11-
### [PRINCIPLE_2_NAME]
12-
<!-- Example: II. CLI Interface -->
13-
[PRINCIPLE_2_DESCRIPTION]
14-
<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
89+
**Rationale**: DCO provides legal protection and clear chain of custody for contributions, which is essential for open-source projects with diverse contributors.
1590

16-
### [PRINCIPLE_3_NAME]
17-
<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
18-
[PRINCIPLE_3_DESCRIPTION]
19-
<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
91+
### VI. Semantic Versioning and Release Discipline
2092

21-
### [PRINCIPLE_4_NAME]
22-
<!-- Example: IV. Integration Testing -->
23-
[PRINCIPLE_4_DESCRIPTION]
24-
<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
93+
Version management MUST follow strict semantic versioning with calendar-based releases:
2594

26-
### [PRINCIPLE_5_NAME]
27-
<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
28-
[PRINCIPLE_5_DESCRIPTION]
29-
<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
95+
- **Project versions** use calendar-based scheme: `YY.MM.PATCH` where April (`.04.`) and October (`.10.`) are stable releases, all other months use `-edge` suffix (e.g., `25.09.0-edge`)
96+
- **Plugin versions** MUST use semantic versioning (`MAJOR.MINOR.PATCH`)
97+
- Version changes MUST be documented in `changelog.txt` files (both project root and per-plugin)
98+
- Breaking changes MUST increment MAJOR version for plugins and be clearly documented
99+
- Release process MUST follow the documented procedure in `CLAUDE.md` including: updating changelogs, version files, running `make releaseInfo`, using `[release]` tag in commit message
30100

31-
## [SECTION_2_NAME]
32-
<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
101+
**Rationale**: Predictable versioning enables users to understand compatibility and stability expectations. Calendar-based versioning for the main project makes release timing transparent, while semantic versioning for plugins enables clear communication of breaking changes.
33102

34-
[SECTION_2_CONTENT]
35-
<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
103+
### VII. Groovy Idioms and Code Standards
36104

37-
## [SECTION_3_NAME]
38-
<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
105+
Code MUST follow Groovy best practices and Nextflow conventions:
39106

40-
[SECTION_3_CONTENT]
41-
<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
107+
- Use Groovy idioms (closures, operator overloading, DSL builders) appropriately
108+
- Follow existing code patterns and conventions from similar modules
109+
- Leverage Groovy's dynamic capabilities judiciously without sacrificing type safety where beneficial
110+
- Use Groovy's `@CompileStatic` where performance is critical or type safety is desired
111+
- AST transformations (in `modules/nextflow`) MUST be well-documented due to their compile-time magic
112+
- Code MUST be formatted consistently (consider CodeNarc configuration in `gradle/codenarc.groovy`)
113+
114+
**Rationale**: Groovy enables powerful DSL capabilities that make Nextflow's language expressive, but requires discipline to maintain readability and debuggability. Consistency across the large codebase improves maintainability.
115+
116+
## Development Workflow
117+
118+
### Build and Development Process
119+
120+
- **Build tool**: Gradle with wrapper (`./gradlew`) is the authoritative build system
121+
- **Quick commands**: Makefile provides convenience targets (`make compile`, `make test`, `make assemble`, `make check`, `make clean`)
122+
- **Development testing**: Use `./launch.sh run script.nf` for testing changes against real workflows without full installation
123+
- **Local installation**: `make install` publishes to Maven local for integration testing
124+
- **Dependency management**: All dependencies MUST be declared in `build.gradle` with explicit versions; use `make deps` to analyze dependency trees
125+
126+
### Git Workflow
127+
128+
- **Branch management**: Work on feature branches, never commit directly to `master`
129+
- **Commit sign-off**: Always use `git commit -s` to add DCO sign-off
130+
- **CI control tags**: Use special commit message tags to control CI behavior:
131+
- `[ci skip]` - Skip CI tests entirely
132+
- `[ci fast]` - Run only unit tests, skip integration tests
133+
- `[e2e stage]` - Run end-to-end tests against Seqera platform staging environment
134+
- `[e2e prod]` - Run end-to-end tests against production platform
135+
- `[release]` - Trigger release automation
136+
- **Pull requests**: Must pass all CI checks, require code review, and have DCO verification
137+
138+
### Architecture Decision Records (ADRs)
139+
140+
- Significant structural and technical decisions MUST be documented as ADRs in the `adr/` directory
141+
- ADRs MUST follow the template format: date prefix + descriptive name (e.g., `20251114-module-system.md`)
142+
- ADRs provide historical context for why architectural decisions were made
143+
- When changing fundamental architecture, review existing ADRs and create new ones documenting the rationale
144+
145+
## Quality Standards
146+
147+
### Code Review Requirements
148+
149+
- All changes MUST go through pull request review
150+
- Reviewers MUST verify:
151+
- Tests are included and passing
152+
- Code follows Groovy idioms and project conventions
153+
- License headers are present
154+
- DCO sign-off is present
155+
- Changes align with modular architecture principles
156+
- Breaking changes are appropriately versioned and documented
157+
158+
### Testing Gates
159+
160+
- `make test` MUST pass before committing locally
161+
- All CI tests MUST pass before merging
162+
- Integration tests MUST be run for changes affecting workflow execution
163+
- Cloud validation tests MUST be run before releases touching cloud provider plugins
164+
- Smoke tests enable rapid iteration but MUST NOT replace full test execution
165+
166+
### Performance and Compatibility
167+
168+
- Target platform: Java 17 runtime compatibility (development uses Java 21 toolchain)
169+
- Performance-critical paths SHOULD be profiled and optimized
170+
- Memory usage SHOULD be monitored for large-scale workflows
171+
- Backward compatibility MUST be maintained for existing DSL features unless a new DSL version is introduced
42172

43173
## Governance
44-
<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
45174

46-
[GOVERNANCE_RULES]
47-
<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
175+
### Amendment Process
176+
177+
This constitution supersedes all other development practices. Amendments require:
178+
179+
1. **Proposal**: Submit amendment proposal via GitHub issue or pull request
180+
2. **Discussion**: Community discussion period (minimum 1 week for major changes)
181+
3. **Approval**: Approval from core maintainers
182+
4. **Documentation**: Update this constitution with version bump following semantic versioning:
183+
- **MAJOR**: Backward incompatible governance changes, principle removal/redefinition
184+
- **MINOR**: New principle added or materially expanded guidance
185+
- **PATCH**: Clarifications, wording improvements, typo fixes
186+
187+
### Compliance and Review
188+
189+
- All pull requests and code reviews MUST verify compliance with these principles
190+
- Deviations from principles MUST be explicitly justified in PR description
191+
- Complexity additions MUST be justified against the "simplicity first" principle
192+
- Constitution compliance is enforced through code review and CI automation where possible
193+
194+
### Ratification and Version History
195+
196+
**Version**: 1.0.0 | **Ratified**: 2025-11-17 | **Last Amended**: 2025-11-17
48197

49-
**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
50-
<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
198+
This constitution was derived from the Nextflow project's documented practices in `CLAUDE.md`, `CONTRIBUTING.md`, and the project's existing architectural patterns. It codifies the development principles that have made Nextflow a successful scientific workflow management system.

0 commit comments

Comments
 (0)