Merge branch 'feature/100-image-limit' into 'develop'

Increase Page Image Limit from 20 to 100

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!441

Author: rstrahan

CHANGELOG.md

@@ -5,6 +5,10 @@ SPDX-License-Identifier: MIT-0
 
 ## [Unreleased]
 
+### Changed
+- Increased page image limit from 20 to 100 across all IDP services (classification, extraction, assessment) to support processing of longer document sections with large context models following recent Amazon Bedrock API limit increases
+  - Resolves #147
+
 ## [0.4.5]
 
 ### Added

VERSION

@@ -1 +1 @@
-0.4.5
+0.4.6-wip1

docs/classification.md

@@ -403,7 +403,7 @@ classification:
 
 For documents with multiple pages, the system automatically handles image limits:
 
-- **Bedrock Limit**: Maximum 20 images per request (automatically enforced)
+- **Bedrock Limit**: Maximum 100 images per request (automatically enforced)
 - **Warning Logging**: System logs warnings when images are truncated due to limits
 - **Smart Handling**: Images are processed in page order, with excess images automatically dropped
 

docs/extraction.md

@@ -334,7 +334,7 @@ extraction:
 For documents with multiple pages, the system provides robust image management:
 
 - **Automatic Pagination**: Images are processed in page order
-- **Bedrock Compliance**: Maximum 20 images per request (automatically enforced)
+- **Bedrock Compliance**: Maximum 100 images per request (automatically enforced)
 - **Smart Truncation**: Excess images are dropped with warning logs
 - **Performance Optimization**: Large image sets are efficiently handled
 
@@ -346,7 +346,7 @@ extraction:
 
     {ATTRIBUTE_NAMES_AND_DESCRIPTIONS}
 
-    Document pages (up to 20 images):
+    Document pages (up to 100 images):
     {DOCUMENT_IMAGE}
 
     Combined text from all pages:

docs/idp-configuration-best-practices.md

@@ -1256,7 +1256,7 @@ classification:
 For documents with multiple pages, the system provides robust image management:
 
 - **Automatic Pagination**: Images are processed in page order
-- **Bedrock Compliance**: Maximum 20 images per request (automatically enforced)
+- **Bedrock Compliance**: Maximum 100 images per request (automatically enforced)
 - **Smart Truncation**: Excess images are dropped with warning logs
 - **Performance Optimization**: Large image sets are efficiently handled
 

lib/idp_common_pkg/idp_common/assessment/README.md

@@ -305,7 +305,7 @@ assess each extracted field:
 
 ### Automatic Image Handling
 - Supports both single and multiple document images
-- Automatically limits to 20 images per Bedrock constraints
+- Automatically limits to 100 images per Bedrock constraints
 - Graceful fallback when images are unavailable
 
 ## Attribute Types and Assessment Formats

lib/idp_common_pkg/idp_common/assessment/granular_service.py

@@ -472,13 +472,13 @@ def _build_cached_prompt_base(
             # Add the images if available
             if page_images:
                 if isinstance(page_images, list):
-                    # Multiple images (limit to 20 as per Bedrock constraints)
-                    if len(page_images) > 20:
+                    # Multiple images (limit to 100 as per Bedrock constraints)
+                    if len(page_images) > 100:
                         logger.warning(
-                            f"Found {len(page_images)} images, truncating to 20 due to Bedrock constraints. "
-                            f"{len(page_images) - 20} images will be dropped."
+                            f"Found {len(page_images)} images, truncating to 100 due to Bedrock constraints. "
+                            f"{len(page_images) - 100} images will be dropped."
                         )
-                    for img in page_images[:20]:
+                    for img in page_images[:100]:
                         content.append(image.prepare_bedrock_image_attachment(img))
                 else:
                     # Single image

lib/idp_common_pkg/idp_common/assessment/service.py

@@ -428,13 +428,13 @@ def _build_content_with_image_placeholder(
         # Add the image if available
         if image_content:
             if isinstance(image_content, list):
-                # Multiple images (limit to 20 as per Bedrock constraints)
-                if len(image_content) > 20:
+                # Multiple images (limit to 100 as per Bedrock constraints)
+                if len(image_content) > 100:
                     logger.warning(
-                        f"Found {len(image_content)} images, truncating to 20 due to Bedrock constraints. "
-                        f"{len(image_content) - 20} images will be dropped."
+                        f"Found {len(image_content)} images, truncating to 100 due to Bedrock constraints. "
+                        f"{len(image_content) - 100} images will be dropped."
                     )
-                for img in image_content[:20]:
+                for img in image_content[:100]:
                     content.append(image.prepare_bedrock_image_attachment(img))
             else:
                 # Single image

lib/idp_common_pkg/idp_common/extraction/agentic_idp.py

@@ -735,20 +735,20 @@ def _prepare_prompt_content(
     else:
         prompt_content = [ContentBlock(text=str(prompt))]
 
-    # Add page images if provided (limit to 20 as per Bedrock constraints)
+    # Add page images if provided (limit to 100 as per Bedrock constraints)
     if page_images:
-        if len(page_images) > 20:
+        if len(page_images) > 100:
             prompt_content.append(
                 ContentBlock(
-                    text=f"There are {len(page_images)} images, initially you'll see 20 of them, use the view_image tool to see the rest."
+                    text=f"There are {len(page_images)} images, initially you'll see 100 of them, use the view_image tool to see the rest."
                 )
             )
 
         prompt_content += [
             ContentBlock(
                 image=ImageContent(format="png", source=ImageSource(bytes=img_bytes))
             )
-            for img_bytes in page_images[:20]
+            for img_bytes in page_images[:100]
         ]
 
     # Add existing data context if provided

lib/idp_common_pkg/idp_common/extraction/service.py

@@ -148,13 +148,13 @@ def _get_default_prompt_content(self) -> list[dict[str, Any]]:
         """
         content = [{"text": task_prompt}]
 
-        # Add image attachments to the content (limit to 20 images as per Bedrock constraints)
+        # Add image attachments to the content (limit to 100 images as per Bedrock constraints)
         if self._page_images:
             logger.info(
                 f"Attaching images to default prompt, for {len(self._page_images)} pages."
             )
-            # Limit to 20 images as per Bedrock constraints
-            for img in self._page_images[:20]:
+            # Limit to 100 images as per Bedrock constraints
+            for img in self._page_images[:100]:
                 content.append(image.prepare_bedrock_image_attachment(img))
 
         return content
@@ -354,7 +354,7 @@ def _build_text_and_image_content(
 
     def _prepare_image_attachments(self, image_content: Any) -> list[dict[str, Any]]:
         """
-        Prepare image attachments for Bedrock, limiting to 20 images.
+        Prepare image attachments for Bedrock, limiting to 100 images.
 
         Args:
             image_content: Single image or list of images
@@ -365,13 +365,13 @@ def _prepare_image_attachments(self, image_content: Any) -> list[dict[str, Any]]
         attachments: list[dict[str, Any]] = []
 
         if isinstance(image_content, list):
-            # Multiple images (limit to 20 as per Bedrock constraints)
-            if len(image_content) > 20:
+            # Multiple images (limit to 100 as per Bedrock constraints)
+            if len(image_content) > 100:
                 logger.warning(
-                    f"Found {len(image_content)} images, truncating to 20 due to Bedrock constraints. "
-                    f"{len(image_content) - 20} images will be dropped."
+                    f"Found {len(image_content)} images, truncating to 100 due to Bedrock constraints. "
+                    f"{len(image_content) - 100} images will be dropped."
                 )
-            for img in image_content[:20]:
+            for img in image_content[:100]:
                 attachments.append(image.prepare_bedrock_image_attachment(img))
         else:
             # Single image