Add Permissions Boundary parameter support to main CloudFormation template

Author: Bob Strahan

memory-bank/activeContext.md

@@ -2,111 +2,85 @@
 
 ## Current Task Focus
 
-**User Question**: Understanding OCR processing architecture for large PDFs (500+ pages) in the IDP accelerator, specifically:
-1. Is OCR processing sequential or distributed by page?
-2. How does Bedrock-only OCR deployment differ?
-3. What parts of the system run sequentially vs distributed?
-4. Handling massive PDFs with hundreds of forms without clear page boundaries
+**Customer Question**: "We are encountering difficulties deploying your IDP stack outside of a sandbox environment due to an organization-wide Service Control Policy (SCP). This policy mandates the attachment of a Permissions Boundary to any new role. Could you please inform us if it is possible to update the CloudFormation template to include a parameterized Permissions Boundary? Without this update, our ability to transition the code to production will be significantly impeded."
 
-## Key Findings
+**Task Status**: Implementation phase - Need to add Permissions Boundary parameter support to CloudFormation templates
 
-### OCR Processing Models
+## Problem Analysis
 
-The IDP accelerator uses **different processing models depending on the pattern**:
+### Current Situation
+- IDP stack creates numerous IAM roles across main template and pattern templates
+- Organization has SCP requiring Permissions Boundary on all new IAM roles
+- Current templates don't support Permissions Boundary configuration
+- Blocking production deployment
 
-#### Pattern 1 (BDA): Sequential Internal Processing
-- **OCR Approach**: Bedrock Data Automation handles everything internally
-- **Processing**: Entire document processed as single unit by BDA service
-- **Concurrency**: Not user-controllable, managed by BDA
-- **Large Documents**: Subject to BDA service limits and timeouts
+### Affected Templates
+- **Main Template**: `template.yaml` - ~15 IAM roles
+- **Pattern 1**: `patterns/pattern-1/template.yaml` - ~8 IAM roles  
+- **Pattern 2**: `patterns/pattern-2/template.yaml` - ~6 roles
+- **Pattern 3**: `patterns/pattern-3/template.yaml` - ~5 roles
+- **Options**: `options/bda-lending-project/template.yaml`, `options/bedrockkb/template.yaml`
 
-#### Pattern 2/3 (Textract + Bedrock): Distributed Page Processing
-- **OCR Approach**: AWS Textract with concurrent page processing
-- **Processing**: **Pages processed in parallel** using ThreadPoolExecutor
-- **Concurrency**: Configurable (default: 20 concurrent workers)
-- **Large Documents**: Optimal for 500+ page documents
+## Solution Design
 
-### Sequential vs Distributed Components
+### Approach: Parameterized Permissions Boundary
+1. **Add optional parameter** to main template for Permissions Boundary ARN
+2. **Conditionally apply boundary** to all IAM roles when provided
+3. **Maintain backward compatibility** for deployments without boundaries
+4. **Cascade parameter** to all nested pattern stacks
 
-#### Sequential Processing:
-1. **Step Functions Workflow**: OCR → Classification → Extraction → Assessment → Summarization
-2. **Classification**: Analyzes all pages to create document boundaries
-3. **BDA Internal Processing**: Everything handled as single unit
+### Implementation Plan
 
-#### Distributed Processing:
-1. **OCR Pages (Pattern 2/3)**: Up to 20 pages processed simultaneously
-2. **Extraction Sections**: Up to 10 document sections processed in parallel
-3. **Independent API Calls**: Each page makes separate Textract calls
+#### Step 1: Main Template Updates (`template.yaml`)
+- Add `PermissionsBoundaryArn` parameter
+- Add `HasPermissionsBoundary` condition
+- Update all IAM role resources with conditional boundary
+- Pass parameter to nested stacks
+- Update CloudFormation interface metadata
 
-## Customer Scenario Analysis
+#### Step 2: Pattern Template Updates
+- Add parameter to each pattern template
+- Update all IAM roles in patterns
+- Maintain consistency across all patterns
 
-### 500+ Page PDF with Multiple Forms
+#### Step 3: Options Template Updates
+- Update BDA lending project template
+- Update Bedrock KB template
 
-**Challenge**: Single PDF containing hundreds of forms without clear page boundaries
+### Key Implementation Details
 
-**Recommended Approach**: Pattern 2 or 3 for optimal performance
-
-**Why Pattern 2/3 is Better**:
-- **Page-Level Parallelism**: 500 pages processed 20 at a time
-- **Memory Efficiency**: Individual pages loaded, not entire document
-- **Fault Tolerance**: Page failures don't stop entire processing
-- **Granular Control**: Can optimize per-page processing
-
-**Classification Strategy**:
-- Use "holistic" classification method to analyze entire document
-- Creates logical sections grouping related pages
-- Handles form boundaries that don't align with page boundaries
-
-## Technical Implementation Details
+**Parameter Definition:**
+```yaml
+PermissionsBoundaryArn:
+  Type: String
+  Default: ""
+  Description: (Optional) ARN of IAM Permissions Boundary policy
+  AllowedPattern: "^(|arn:aws:iam::[0-9]{12}:policy/.+)$"
+```
 
-### OCR Service Configuration for Large Documents
+**Condition:**
+```yaml
+HasPermissionsBoundary: !Not [!Equals [!Ref PermissionsBoundaryArn, ""]]
+```
 
+**Role Update Pattern:**
 ```yaml
-ocr:
-  backend: "textract"
-  max_workers: 20  # Increase for more parallelism
-  image:
-    dpi: 150      # Balance quality vs processing time
-    target_width: 1024
-    target_height: 1024
-  features:
-    - name: "LAYOUT"
-    - name: "TABLES" 
-    - name: "FORMS"
+SomeRole:
+  Type: AWS::IAM::Role
+  Properties:
+    # existing properties...
+    PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
 ```
 
-### Processing Flow for Large PDFs
-
-1. **Document Load**: PyMuPDF loads PDF structure
-2. **Page Distribution**: ThreadPoolExecutor creates 20 concurrent workers
-3. **Parallel OCR**: Each page processed independently via Textract
-4. **Result Assembly**: Pages sorted and combined into document structure
-5. **Classification**: Holistic analysis creates logical document sections
-6. **Parallel Extraction**: Sections processed concurrently (MaxConcurrency: 10)
-
-## Performance Implications
-
-### For 500-Page Document:
-- **Pattern 1 (BDA)**: Single job, BDA-managed processing
-- **Pattern 2/3**: ~25 batches of 20 pages each, highly parallelized
-
-### Bottlenecks to Consider:
-1. **Textract Rate Limits**: May need to adjust max_workers
-2. **Memory Usage**: 20 concurrent pages require significant memory
-3. **S3 Operations**: Parallel uploads/downloads for page results
-4. **Lambda Timeouts**: Ensure sufficient timeout for large documents
-
-## Next Steps and Considerations
-
-### For Customer Implementation:
-1. **Choose Pattern 2 or 3** for large document processing
-2. **Configure max_workers** based on Textract limits and memory
-3. **Use holistic classification** to handle form boundaries
-4. **Monitor memory usage** during processing
-5. **Consider document splitting** if single PDF approach is problematic
-
-### Optimization Opportunities:
-- **Adaptive Concurrency**: Adjust workers based on document size
-- **Progressive Processing**: Start classification while OCR continues
-- **Caching Strategy**: Cache page images for reprocessing
-- **Error Recovery**: Implement page-level retry with exponential backoff
+## Benefits
+- **SCP Compliance**: Satisfies organizational requirements
+- **Backward Compatible**: Existing deployments unaffected
+- **Flexible**: Works with any Permissions Boundary policy
+- **Comprehensive**: Covers all IAM roles across all components
+
+## Next Steps
+1. Implement main template changes
+2. Update all pattern templates
+3. Update options templates
+4. Test deployment scenarios
+5. Document usage examples

template.yaml

@@ -290,6 +290,18 @@ Parameters:
     Description: Maximum acceptable execution time in milliseconds before alerting (default 300000 = 300 seconds)
     MinValue: 1000
 
+  # Security Configuration
+  PermissionsBoundaryArn:
+    Type: String
+    Default: ""
+    Description: >-
+      (Optional) ARN of an existing IAM Permissions Boundary policy to attach to all IAM roles.
+      Required by some organizations with Service Control Policies (SCPs).
+      Format: arn:aws:iam::account-id:policy/policy-name
+      Leave blank if no Permissions Boundary is required.
+    AllowedPattern: "^(|arn:aws:iam::[0-9]{12}:policy/.+)$"
+    ConstraintDescription: Must be empty or a valid IAM policy ARN
+
   # Logging configuration
   LogLevel:
     Type: String
@@ -428,6 +440,7 @@ Conditions:
   ShouldCreateDocumentKnowledgeBase: !Equals [!Ref DocumentKnowledgeBase, "BEDROCK_KNOWLEDGE_BASE (Create)"]
   ShouldUseDocumentKnowledgeBase: !Condition ShouldCreateDocumentKnowledgeBase
   DocumentSectionsCrawlerScheduleEnabled: !Not [!Equals [!Ref DocumentSectionsCrawlerFrequency, "Manual"]]
+  HasPermissionsBoundary: !Not [!Equals [!Ref PermissionsBoundaryArn, ""]]
 
 
 Metadata:
@@ -492,6 +505,10 @@ Metadata:
         Parameters:
           - BedrockGuardrailId
           - BedrockGuardrailVersion
+      - Label:
+          default: "Security Configuration"
+        Parameters:
+          - PermissionsBoundaryArn
       - Label:
           default: "General Configuration"
         Parameters:
@@ -550,6 +567,8 @@ Metadata:
         default: "Bedrock Guardrail Id"
       BedrockGuardrailVersion:
         default: "Bedrock Guardrail Version"
+      PermissionsBoundaryArn:
+        default: "Permissions Boundary ARN"
       MaxConcurrentWorkflows:
         default: "Maximum Concurrent Workflows"
       DataRetentionInDays:
@@ -828,6 +847,7 @@ Resources:
           - "s3://${ConfigurationBucket}/config_library/pattern-1/${ConfigPath}/config.yaml"
           - ConfigPath: !FindInMap [Pattern1ConfigurationMap, !Ref Pattern1Configuration, ConfigPath]
         ConfigLibraryHash: "<CONFIG_LIBRARY_HASH_TOKEN>"
+        PermissionsBoundaryArn: !Ref PermissionsBoundaryArn
         SageMakerA2IReviewPortalURL: !If
           - IsPattern1HITLEnabled
           - !GetAtt WorkforceURLResource.PortalURL
@@ -867,6 +887,7 @@ Resources:
           - "s3://${ConfigurationBucket}/config_library/pattern-2/${ConfigPath}/config.yaml"
           - ConfigPath: !FindInMap [Pattern2ConfigurationMap, !Ref Pattern2Configuration, ConfigPath]
         ConfigLibraryHash: "<CONFIG_LIBRARY_HASH_TOKEN>"
+        PermissionsBoundaryArn: !Ref PermissionsBoundaryArn
 
   PATTERN3STACK:
     DependsOn:
@@ -901,6 +922,7 @@ Resources:
           - "s3://${ConfigurationBucket}/config_library/pattern-3/${ConfigPath}/config.yaml"
           - ConfigPath: !FindInMap [Pattern3ConfigurationMap, !Ref Pattern3Configuration, ConfigPath]
         ConfigLibraryHash: "<CONFIG_LIBRARY_HASH_TOKEN>"
+        PermissionsBoundaryArn: !Ref PermissionsBoundaryArn
 
   ##########################################################################
   # Encryption key
@@ -1671,6 +1693,7 @@ Resources:
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: DocumentSectionsCrawlerS3Access
           PolicyDocument:
@@ -3504,6 +3527,7 @@ Resources:
                 "cognito-identity.amazonaws.com:aud": !Ref IdentityPool
               "ForAnyValue:StringLike":
                 "cognito-identity.amazonaws.com:amr": authenticated
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: S3
           PolicyDocument:
@@ -3703,6 +3727,7 @@ Resources:
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: CognitoClientUpdaterPolicy
           PolicyDocument:
@@ -3799,6 +3824,7 @@ Resources:
             Principal:
               Service: sagemaker.amazonaws.com
             Action: sts:AssumeRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: A2IFlowDefinitionAccess
           PolicyDocument:
@@ -4046,6 +4072,7 @@ Resources:
     Properties:
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSAppSyncPushToCloudWatchLogs
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       AssumeRolePolicyDocument:
         Version: 2012-10-17
         Statement:
@@ -4209,6 +4236,7 @@ Resources:
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSAppSyncPushToCloudWatchLogs
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: DynamoDBAccess
           PolicyDocument: