feat: graduate Notes API to dedicated notes.miloapis.com group#488
feat: graduate Notes API to dedicated notes.miloapis.com group#488OscarLlamas6 wants to merge 10 commits intomainfrom
Conversation
- Create new notes.miloapis.com API group with Note (namespaced) and ClusterNote (cluster-scoped) - Implement controllers with PolicyBinding support for both resource types - Add validation and mutation webhooks for Note and ClusterNote - Generate CRDs for notes.miloapis.com/v1alpha1 - Configure ProtectedResources for IAM integration - Add domain note examples demonstrating all use cases - Remove Notes from crm.miloapis.com (graduated to dedicated group) - Clean up CRM API group structure for future CRM-specific resources - Update all kustomizations and configurations This allows Notes to be attached to any resource (including networking.datumapis.com/Domain) while keeping CRM API group available for future CRM-specific resources like Leads, Opportunities, and Accounts. Resolves: #[issue-number]
| return ctrl.Result{}, nil | ||
| } | ||
|
|
||
| func (r *ClusterNoteController) ensureCreatorEditorPolicyBinding(ctx context.Context, clusterNote *notesv1alpha1.ClusterNote, creator *iamv1alpha1.User) (bool, string, error) { |
There was a problem hiding this comment.
Both controllers (clusternote and note) share most of the logic. While it’s beneficial to have separate controllers, I would suggest sharing all of the functions that would be common to both.
JoseSzycho
left a comment
There was a problem hiding this comment.
Overall, great work.
Just left some comments about sharing functionalities and setting up the resources owners.
… collection Controllers: - Extract common PolicyBinding logic to shared.go - Add NoteResource interface implemented by Note and ClusterNote - Share ensureCreatorEditorPolicyBinding() and isPolicyBindingReady() functions - Eliminate code duplication between Note and ClusterNote controllers Webhooks: - Add owner reference to subject resource in Defaulter for automatic garbage collection - Note: sets owner reference if subject is in same namespace - ClusterNote: sets owner reference if subject is cluster-scoped - Non-blocking: webhook succeeds even if owner reference fails to set This ensures notes are automatically deleted when their subject resource is deleted, and reduces code duplication by sharing common controller logic.
Thanks for the feedback @JoseSzycho ! I've implemented both suggestions. Let me know your feedback again please |
JoseSzycho
left a comment
There was a problem hiding this comment.
Looks great! I really like the use of gvk, so the notes can be used for any resources.
I’ve left a few minor comments.
What do you think if we rename Note to NamespaceNote? This way, API consumers won’t be confused about when to use each.
Please add some unit tests for the webhooks and end-to-end tests for both note types. You should be able to verify that the garbage collection works and that the creatorRef is correctly set.
Webhook Validations: - Note: validate subject is namespaced and in same namespace - ClusterNote: validate subject is cluster-scoped (no namespace) - Both: fail webhook if owner reference cannot be set (ensures GC works) Unit Tests: - Add comprehensive test coverage for Note and ClusterNote webhooks - Test mutator (defaulter) with owner reference setup scenarios - Test validator with valid/invalid subject references - All 17 tests passing Local Testing: - Verified validations work correctly in local kind cluster - Confirmed owner references are set properly - Tested garbage collection for both Note and ClusterNote - All scenarios working as expected This ensures proper garbage collection and prevents invalid configurations that would break the owner reference mechanism.
|
@JoseSzycho Thanks for the detailed feedback! I've implemented all the suggestions. Here's what changed: ClusterNote validation - Now validates that if clusterNote.Spec.SubjectRef.Namespace != "" {
return Invalid("ClusterNote can only reference cluster-scoped resources")
}Note validation - Validates that subject is namespaced AND in the same namespace: if note.Spec.SubjectRef.Namespace == "" {
return Invalid("Note can only reference namespaced resources")
} else if note.Spec.SubjectRef.Namespace != note.Namespace {
return Invalid("Note must be in the same namespace as its subject")
}Owner reference enforcement - Changed the webhook to fail if owner reference can't be set. This ensures GC always works as intended. The webhook will fail if:
This is a fail-fast approach - if we can't guarantee GC will work, we don't create the note. Unit TestsAdded comprehensive tests for both webhooks - 17 test cases total, all passing ✅ Tests cover:
Local TestingSpun up a local kind cluster with Milo and tested both scenarios: ClusterNote + Organization: $ kubectl get clusternote clusternote-gc-test -o yaml
ownerReferences:
- apiVersion: resourcemanager.miloapis.com/v1alpha1
kind: Organization
name: test-org-gc
$ kubectl delete organization test-org-gc
$ kubectl get clusternote clusternote-gc-test
Error: NotFound # ✅ GC worked!Note + Group: $ kubectl get note note-gc-test -n test-notes -o yaml
ownerReferences:
- apiVersion: iam.miloapis.com/v1alpha1
kind: Group
name: test-group-gc
$ kubectl delete group test-group-gc -n test-notes
$ kubectl get note note-gc-test -n test-notes
Error: NotFound # ✅ GC worked!Also verified the validations work - tried creating invalid configs and they were all rejected with clear error messages. |
@JoseSzycho I'd prefer to keep it as |
…issing test files This commit addresses a critical bug in the Notes controller and restores missing test files that were accidentally deleted during the Notes API graduation. **Problem:** The Notes controller attempted to set owner references from Notes (in any namespace) to PolicyBindings (always in milo-system), which Kubernetes disallows. This caused the controller to fail with: "cross-namespace owner references are disallowed, owner's namespace X, obj's namespace milo-system" **Solution:** - Removed cross-namespace owner references from PolicyBindings - Implemented finalizers on Note and ClusterNote resources - Added labels to PolicyBindings for ownership tracking (note-name, note-namespace, note-uid) - Implemented manual cleanup of PolicyBindings via finalizers when Notes are deleted - Added helper functions: containsString(), removeString(), cleanupPolicyBindings() **Changes:** - internal/controllers/notes/shared.go: Removed SetOwnerReference, added cleanup logic - internal/controllers/notes/note_controller.go: Added finalizer handling - internal/controllers/notes/clusternote_controller.go: Added finalizer handling - config/controller-manager/overlays/core-control-plane/rbac/role.yaml: Added finalizers and policybindings permissions **Problem:** File [test/crm/note-contact-lifecycle/03-test-notes.yaml](cci:7://file:///home/oscar/Desktop/Trabajo/DATUM/Repositories/milo/test/crm/note-contact-lifecycle/03-test-notes.yaml:0:0-0:0) was accidentally deleted during the Notes API graduation to notes.miloapis.com, causing the test suite to fail. **Solution:** - Recreated 03-test-notes.yaml with correct notes.miloapis.com/v1alpha1 API - Updated test-crm-note-2 from Note to ClusterNote (references cluster-scoped User) - Updated chainsaw-test.yaml to use notes.miloapis.com/v1alpha1 throughout - Simplified test to focus on owner references and garbage collection - Removed status.createdBy verification (controller now works without PolicyBinding errors) ✅ Unit tests: All webhook tests passing ✅ E2E test: crm-note-contact-lifecycle passing (6.28s) ✅ Manual verification: Garbage collection working for both Note and ClusterNote ✅ Controller logs: No more cross-namespace owner reference errors Fixes the test failure reported in CI and resolves the underlying controller bug that prevented Notes from functioning correctly in production environments.
… references Based on team feedback, simplified the PolicyBinding creation strategy to use Kubernetes' native garbage collection instead of manual finalizers. ## Changes **PolicyBinding Namespace Strategy:** - **Note (namespaced):** PolicyBindings now created in the same namespace as the Note - **ClusterNote:** PolicyBindings created in milo-system (both cluster-scoped) This allows owner references to work correctly without cross-namespace issues. **Removed:** - Finalizer logic from Note and ClusterNote controllers - Manual cleanup functions (cleanupPolicyBindings, containsString, removeString) - Finalizer-related labels (note-name, note-namespace, note-uid) - RBAC permissions for finalizers and policybinding delete/list **Modified:** - internal/controllers/notes/shared.go: PolicyBindings created in Note's namespace or milo-system for ClusterNote - internal/controllers/notes/note_controller.go: Removed finalizer handling - internal/controllers/notes/clusternote_controller.go: Removed finalizer handling - config/controller-manager/overlays/core-control-plane/rbac/role.yaml: Removed finalizer permissions ## Benefits ✅ Simpler code - no finalizer logic needed ✅ Native Kubernetes garbage collection works automatically ✅ Owner references work correctly (same namespace or both cluster-scoped) ✅ Easier to understand and maintain ## Verification ✅ E2E test: crm-note-contact-lifecycle passing (6.14s) ✅ Manual test: PolicyBinding created in Note's namespace (not milo-system) ✅ Manual test: Owner reference correctly set to Note ✅ Manual test: Garbage collection works - PolicyBinding deleted when Note deleted Addresses feedback from @scotwells and @JoseSzycho
- Remove obsolete crm-note-* roles (viewer, creator, editor, creator-editor, admin) - Create new notes-* roles for notes.miloapis.com API group - Update roles to include permissions for both Note and ClusterNote resources - Update config/roles/kustomization.yaml to reference new role files - Update controller-manager flag default from crm-note-creator-editor to notes-creator-editor This change aligns with the migration of Note resources from crm.miloapis.com to the dedicated notes.miloapis.com API group, which now supports both namespace-scoped Notes and cluster-scoped ClusterNotes.
This allows Notes to be attached to any resource (including networking.datumapis.com/Domain) while keeping CRM API group available for future CRM-specific resources like Leads, Opportunities, and Accounts.
Needed for datum-cloud/enhancements#501