Merge branch 'fix/remove-image-limits' into 'develop'

Remove image count limits for Bedrock API calls across IDP services

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!443

Author: rstrahan

CHANGELOG.md

@@ -6,7 +6,7 @@ SPDX-License-Identifier: MIT-0
 ## [Unreleased]
 
 ### Changed
-- Increased page image limit from 20 to 100 across all IDP services (classification, extraction, assessment) to support processing of longer document sections with large context models following recent Amazon Bedrock API limit increases
+- Removed page image limit entirely across all IDP services (classification, extraction, assessment) following Amazon Bedrock API removal of image count restrictions. The system now processes all document pages without artificial truncation, with info logging to track image counts for monitoring purposes.
   - Resolves #147
 
 ## [0.4.5]

docs/classification.md

@@ -401,11 +401,11 @@ classification:
 
 ### Multi-Page Documents
 
-For documents with multiple pages, the system automatically handles image limits:
+For documents with multiple pages, the system provides comprehensive image support:
 
-- **Bedrock Limit**: Maximum 100 images per request (automatically enforced)
-- **Warning Logging**: System logs warnings when images are truncated due to limits
-- **Smart Handling**: Images are processed in page order, with excess images automatically dropped
+- **No Image Limits**: All document pages are processed following Bedrock API removal of image count restrictions
+- **Info Logging**: System logs image counts for monitoring and debugging purposes
+- **Automatic Pagination**: Images are processed in page order for all pages
 
 ## Setting Up Few Shot Examples in Pattern 2
 

docs/extraction.md

@@ -334,9 +334,9 @@ extraction:
 For documents with multiple pages, the system provides robust image management:
 
 - **Automatic Pagination**: Images are processed in page order
-- **Bedrock Compliance**: Maximum 100 images per request (automatically enforced)
-- **Smart Truncation**: Excess images are dropped with warning logs
-- **Performance Optimization**: Large image sets are efficiently handled
+- **No Image Limits**: All document pages are included following Bedrock API removal of image count restrictions
+- **Comprehensive Processing**: The system processes documents of any length without truncation
+- **Performance Optimization**: Efficient handling of large image sets with info logging
 
 ```yaml
 # Example configuration for multi-page invoices
@@ -346,7 +346,7 @@ extraction:
 
     {ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
 
-    Document pages (up to 100 images):
+    Document pages (all pages included):
     {DOCUMENT_IMAGE}
 
     Combined text from all pages:

docs/idp-configuration-best-practices.md

@@ -1253,12 +1253,12 @@ classification:
 
 ### Multi-Page Document Handling
 
-For documents with multiple pages, the system provides robust image management:
+For documents with multiple pages, the system provides comprehensive image support:
 
 - **Automatic Pagination**: Images are processed in page order
-- **Bedrock Compliance**: Maximum 100 images per request (automatically enforced)
-- **Smart Truncation**: Excess images are dropped with warning logs
-- **Performance Optimization**: Large image sets are efficiently handled
+- **No Image Limits**: All document pages are processed following Bedrock API removal of image count restrictions
+- **Info Logging**: System logs image counts for monitoring purposes
+- **Comprehensive Processing**: Documents of any length are fully processed
 
 ### Best Practices for Image Processing
 

lib/idp_common_pkg/idp_common/assessment/README.md

@@ -305,8 +305,9 @@ assess each extracted field:
 
 ### Automatic Image Handling
 - Supports both single and multiple document images
-- Automatically limits to 100 images per Bedrock constraints
+- Processes all document pages without image count restrictions
 - Graceful fallback when images are unavailable
+- Info logging for image count monitoring
 
 ## Attribute Types and Assessment Formats
 

lib/idp_common_pkg/idp_common/assessment/granular_service.py

@@ -472,13 +472,11 @@ def _build_cached_prompt_base(
             # Add the images if available
             if page_images:
                 if isinstance(page_images, list):
-                    # Multiple images (limit to 100 as per Bedrock constraints)
-                    if len(page_images) > 100:
-                        logger.warning(
-                            f"Found {len(page_images)} images, truncating to 100 due to Bedrock constraints. "
-                            f"{len(page_images) - 100} images will be dropped."
-                        )
-                    for img in page_images[:100]:
+                    # Multiple images - no limit with latest Bedrock API
+                    logger.info(
+                        f"Attaching {len(page_images)} images to granular assessment prompt"
+                    )
+                    for img in page_images:
                         content.append(image.prepare_bedrock_image_attachment(img))
                 else:
                     # Single image

lib/idp_common_pkg/idp_common/assessment/service.py

@@ -428,13 +428,11 @@ def _build_content_with_image_placeholder(
         # Add the image if available
         if image_content:
             if isinstance(image_content, list):
-                # Multiple images (limit to 100 as per Bedrock constraints)
-                if len(image_content) > 100:
-                    logger.warning(
-                        f"Found {len(image_content)} images, truncating to 100 due to Bedrock constraints. "
-                        f"{len(image_content) - 100} images will be dropped."
-                    )
-                for img in image_content[:100]:
+                # Multiple images - no limit with latest Bedrock API
+                logger.info(
+                    f"Attaching {len(image_content)} images to assessment prompt"
+                )
+                for img in image_content:
                     content.append(image.prepare_bedrock_image_attachment(img))
             else:
                 # Single image

lib/idp_common_pkg/idp_common/extraction/agentic_idp.py

@@ -735,20 +735,18 @@ def _prepare_prompt_content(
     else:
         prompt_content = [ContentBlock(text=str(prompt))]
 
-    # Add page images if provided (limit to 100 as per Bedrock constraints)
+    # Add page images if provided - no limit with latest Bedrock API
     if page_images:
-        if len(page_images) > 100:
-            prompt_content.append(
-                ContentBlock(
-                    text=f"There are {len(page_images)} images, initially you'll see 100 of them, use the view_image tool to see the rest."
-                )
-            )
+        logger.info(
+            "Attaching images to agentic extraction prompt",
+            extra={"image_count": len(page_images)},
+        )
 
         prompt_content += [
             ContentBlock(
                 image=ImageContent(format="png", source=ImageSource(bytes=img_bytes))
             )
-            for img_bytes in page_images[:100]
+            for img_bytes in page_images
         ]
 
     # Add existing data context if provided

lib/idp_common_pkg/idp_common/extraction/service.py

@@ -148,13 +148,12 @@ def _get_default_prompt_content(self) -> list[dict[str, Any]]:
         """
         content = [{"text": task_prompt}]
 
-        # Add image attachments to the content (limit to 100 images as per Bedrock constraints)
+        # Add image attachments to the content - no limit with latest Bedrock API
         if self._page_images:
             logger.info(
-                f"Attaching images to default prompt, for {len(self._page_images)} pages."
+                f"Attaching {len(self._page_images)} images to default extraction prompt"
             )
-            # Limit to 100 images as per Bedrock constraints
-            for img in self._page_images[:100]:
+            for img in self._page_images:
                 content.append(image.prepare_bedrock_image_attachment(img))
 
         return content
@@ -354,7 +353,7 @@ def _build_text_and_image_content(
 
     def _prepare_image_attachments(self, image_content: Any) -> list[dict[str, Any]]:
         """
-        Prepare image attachments for Bedrock, limiting to 100 images.
+        Prepare image attachments for Bedrock - no image limit.
 
         Args:
             image_content: Single image or list of images
@@ -365,13 +364,9 @@ def _prepare_image_attachments(self, image_content: Any) -> list[dict[str, Any]]
         attachments: list[dict[str, Any]] = []
 
         if isinstance(image_content, list):
-            # Multiple images (limit to 100 as per Bedrock constraints)
-            if len(image_content) > 100:
-                logger.warning(
-                    f"Found {len(image_content)} images, truncating to 100 due to Bedrock constraints. "
-                    f"{len(image_content) - 100} images will be dropped."
-                )
-            for img in image_content[:100]:
+            # Multiple images - no limit with latest Bedrock API
+            logger.info(f"Attaching {len(image_content)} images to extraction prompt")
+            for img in image_content:
                 attachments.append(image.prepare_bedrock_image_attachment(img))
         else:
             # Single image