Add AWS China Regions Support (cn-north-1 and cn-northwest-1)

[crogerc]

Author(s)

Roger Zhang (@crogerc) - zxiaogon@amazon.com

Summary

Add support for AWS China regions (Beijing and Ningxia) to enable deployment of the Multi-Provider Generative AI Gateway in the AWS China partition (aws-cn).

Motivation

AWS China regions are operated independently by local partners (Sinnet for Beijing, NWCD for Ningxia) and constitute a separate partition from global AWS regions. This creates several key differences:

• Separate partition: Uses aws-cn partition instead of aws, requiring different ARN formats
• Independent account system: Requires separate AWS China accounts and credentials
• Regulatory compliance: Subject to Chinese data sovereignty and ICP filing requirements
• Service availability: Some AWS services may have different availability
• Market demand: Many Chinese enterprises and organizations require data to remain within China borders

Currently, this project only supports global AWS regions, preventing Chinese customers from deploying this solution to meet their compliance and data residency requirements.

Proposal

Extend the project to support AWS China regions by adding region-specific configurations and updating deployment scripts to handle the China partition.

User Experience

Users deploying in AWS China regions will:

1. Use the same deployment process as global regions
2. Have deployments that respect China partition ARN formats
3. Receive clear documentation about how to connect Global BedRock endpoint
4. Receive clear documentation about China-specific requirements (ICP filing, etc.)

Technical Implementation

1. Create Region-Specific Configuration Files

Add two new configuration files following the existing pattern:

• config/default-config-cn-north-1.yaml (Beijing region)
• config/default-config-cn-northwest-1.yaml (Ningxia region)

These files should include:

• Available Amazon Bedrock models in China regions
• Proper model naming conventions for China partition
• Any region-specific model configurations

2. Update Deployment Scripts

Modify deploy.sh to:

• Detect China regions (cn-north-1, cn-northwest-1)
• Handle aws-cn partition in ARN construction
• Adjust service endpoints for China partition where necessary
• Ensure proper region detection logic

3. Update Terraform Configurations

Update litellm-terraform-stack and related Terraform modules to:

• Support aws-cn partition ARN formats
• Handle China-specific service endpoints
• Ensure all used AWS services are available in China regions
• Add appropriate provider configurations for China partition

4. Verify Service Availability

Confirm the following services are available and properly configured in China regions:

• Amazon ECS / Amazon EKS
• Amazon RDS
• Amazon ElastiCache (Redis)
• Application Load Balancer
• Amazon CloudFront (note: different behavior in China)
• AWS WAF
• AWS Certificate Manager
• Amazon S3
• AWS Secrets Manager

5. Update Documentation

• Add cn-north-1 and cn-northwest-1 to the "Supported AWS Regions" table in README.md
• Document China-specific requirements and considerations
• Add notes about ICP filing requirements for public-facing deployments
• Document any service limitations or differences in China regions

Alternatives Considered

Alternative 1: Separate Fork for China

Create a separate fork specifically for China regions.

• Pros: Complete independence, easier to customize
• Cons: Maintenance burden, fragmentation, duplicate effort

Alternative 2: Manual Configuration

Let users manually adapt configurations for China.

• Pros: No code changes needed
• Cons: Poor user experience, error-prone, not scalable

Alternative 3: Current Approach (Recommended)

Integrate China support into the main project.

• Pros: Unified codebase, better maintenance, consistent experience
• Cons: Slightly increased complexity

Benefits

1. Market Expansion: Enables Chinese customers to use this solution
2. Compliance: Helps organizations meet Chinese data residency requirements
3. Completeness: Demonstrates comprehensive AWS global coverage
4. Community: Attracts Chinese developer community contributions
5. Best Practice: Shows proper multi-partition AWS architecture

Drawbacks

1. Testing Complexity: Requires AWS China account for proper testing
2. Maintenance: Need to track China-specific service updates
3. Documentation: Additional documentation for China-specific requirements
4. Limited Access: Not all contributors may have access to AWS China for testing

Required Resources

• Development Time: Estimated 2-3 days for implementation
• Testing: Access to AWS China account (cn-north-1 or cn-northwest-1)
• Documentation: 1 day for comprehensive documentation updates
• Review: Maintainer time for code review and validation

Dependencies

1. Access to AWS China account for testing and validation
2. Knowledge of available Bedrock models in China regions
3. Understanding of China partition ARN format differences
4. Familiarity with ICP and Chinese compliance requirements (for documentation)

Testing Plan

1. Unit Testing: Verify configuration file syntax and structure
2. Integration Testing:
   • Deploy to cn-north-1 region
   • Deploy to cn-northwest-1 region
   • Test all core features (model access, authentication, logging, etc.)
3. Validation:
   • Verify Bedrock model access
   • Test ECS/EKS deployment
   • Confirm proper ARN handling
   • Validate CloudWatch logging
   • Test S3 log bucket creation

Rollout Plan

1. Phase 1: Create configuration files and update deployment scripts
2. Phase 2: Test in development environment with AWS China account
3. Phase 3: Update documentation with China-specific guidance
4. Phase 4: Submit PR for review
5. Phase 5: Address feedback and merge
6. Phase 6: Announce China region support in release notes

Documentation

Required documentation updates:

1. README.md:
   • Add cn-north-1 and cn-northwest-1 to supported regions table
   • Add "AWS China Regions" section with special considerations
2. Deployment Guide:
   • Document China-specific deployment steps
   • Explain ICP filing requirements for public deployments
   • Note any service limitations
3. Configuration Guide:
   • Explain partition-specific configurations

Additional Notes

I am willing to contribute this feature and can:

• Create the necessary configuration files
• Update deployment scripts
• Test in AWS China regions (if I have access)
• Write comprehensive documentation
• Respond to code review feedback

Please let me know if this enhancement aligns with the project's roadmap and if there are any specific requirements or preferences for the implementation.