Feature/service tiers #158
Draft
+1,131
−49
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement configurable service tier selection (Priority, Standard, Flex) for all Bedrock API calls to optimize performance and cost. Features: - Global service tier configuration with operation-specific overrides - Support for all operations: OCR, Classification, Extraction, Assessment, Summarization - Web UI components for service tier selection - CLI parameters for service tier specification - Automatic normalization of 'standard' to 'default' for API compatibility - Comprehensive unit tests (7/7 passing) - Full documentation in docs/service-tiers.md Backend Changes: - BedrockClient: Added service_tier parameter with validation and normalization - Configuration models: Added service_tier fields to all operation configs - Services: All services read and pass service_tier to Bedrock API calls - Fallback chain: CLI → Operation Config → Global Config → 'standard' Frontend Changes: - GlobalServiceTierSection component for global tier selection - OperationServiceTierField component for operation overrides - Service tier constants and help text - Integrated into ConfigurationLayout Configuration: - All pattern configs updated with service_tier settings - Global default: 'standard' - Operation overrides: null (use global default) Testing: - 7 unit tests covering all service tier scenarios - All tests passing - All lint checks passing (ruff) Backward Compatibility: - Fully backward compatible - Optional parameter, defaults to 'standard' - Existing configurations work without modification Closes: Service tier support implementation
Update all IDP services to read and pass service_tier parameter to Bedrock API. Update all default configuration files with service_tier settings. Service Integration: - OCR: Reads service_tier from ocr.service_tier or global config - Classification: Includes service_tier in config dict - Extraction: Reads service_tier with fallback logic - Assessment: Reads and passes service_tier to Bedrock - Granular Assessment: Propagates service_tier through parallel/sequential processing - Summarization: Includes service_tier in config dict Configuration Updates: - Pattern 1: Added global and operation-specific service_tier settings - Pattern 2: Added global and operation-specific service_tier settings - Pattern 3: Added global and operation-specific service_tier settings - All configs include explanatory comments CLI Updates: - Added --service-tier parameter to deploy command - Added --service-tier parameter to run-inference command - Validates against: priority, standard, flex Quality: - All ruff lint checks passing - Type hints complete - Docstrings updated - Backward compatible
- Fix indentation in ConfigurationLayout.jsx onSchemaChange callback - Add PropTypes validation to GlobalServiceTierSection component - Add PropTypes validation to OperationServiceTierField component - Fix formatting in configTypes.js SERVICE_TIER_HELP_TEXT
…r boto3 API spec
- Changed serviceTier from string to dict format: {'type': 'flex'}
- Updated BedrockClient to pass serviceTier as {'type': value}
- Updated unit tests to verify correct dictionary format
- Added integration tests for Nova 2 Lite with all service tiers
- Created standalone test scripts for verification
- All tests pass with real AWS Bedrock API calls
- Verified no incorrect usage of 'service_tier' in boto3 calls
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #157
Added support to specify service_tier keyword for supported models
This is a work in progress and has not been deploy-tested yet.