Merge branch 'fix/pdf-resolution' into 'develop'

Fix/pdf resolution

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!228

Author: rstrahan

CHANGELOG.md

@@ -14,6 +14,11 @@ SPDX-License-Identifier: MIT-0
   - Fixed view toggle behavior - switching between views no longer closes the viewer window
   - Reordered view buttons to: Markdown View, Text Confidence View, Text View for better user experience
 
+- **Enhanced OCR DPI Configuration for PDF files**
+  - DPI for PDF image conversion is now configurable in the configuration editor under OCR image processing settings
+  - Default DPI improved from 96 to 150 DPI for better default quality and OCR accuracy
+  - Configurable through Web UI without requiring code changes or redeployment
+
 ### Changed
 - **Converted text confidence data format from JSON to markdown table for improved readability and reduced token usage**
   - Removed unnecessary "page_count" field
@@ -26,8 +31,13 @@ SPDX-License-Identifier: MIT-0
   - Aligned with classification service pattern for better consistency across IDP services
   - Backward compatibility maintained - old parameter pattern still supported with deprecation warning
   - Updated all lambda functions and notebooks to use new simplified pattern
+- Removed fixed image target_height and target_width from default configurations, so images are processed in original resolution by default.
+
 
 ### Fixed
+- **Fixed Image Resizing Behavior for High-Resolution Documents**
+  - Fixed issue where empty strings in image configuration were incorrectly resizing images to default 951x1268 pixels instead of preserving original resolution
+  - Empty strings (`""`) in `target_width` and `target_height` configuration now preserve original document resolution for maximum processing accuracy
 - Fixed issue where PNG files were being unnecessarily converted to JPEG format and resized to lower resolution with lost quality
 - Fixed issue where PNG and JPG image files were not rendering inline in the Document Details page
 - Fixed issue where PDF files were being downloaded instead of displayed inline

VERSION

@@ -1 +1 @@
-0.3.8-wip4
+0.3.8-wip6

config_library/pattern-2/bank-statement-sample/config.yaml

@@ -10,8 +10,9 @@ ocr:
   features:
     - name: LAYOUT
   image:
-    target_width: '951'
-    target_height: '1268'
+    dpi: '150'
+    target_width: ''
+    target_height: ''
 classes:
   - name: Bank Statement
     description: Monthly bank account statement
@@ -68,8 +69,8 @@ classes:
         attributeType: list
 classification:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   top_p: '0.1'
   max_tokens: '4096'
   top_k: '5'
@@ -210,8 +211,8 @@ classification:
   classificationMethod: textbasedHolisticClassification
 extraction:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   top_p: '0.1'
   max_tokens: '10000'
   top_k: '5'
@@ -368,8 +369,8 @@ summarization:
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   granular:
     enabled: true
     max_workers: "20"

config_library/pattern-2/default/config.yaml

@@ -12,8 +12,9 @@ ocr:
     - name: TABLES
     - name: SIGNATURES
   image:
-    target_width: '951'
-    target_height: '1268'
+    dpi: '150'
+    target_width: ''
+    target_height: ''
 classes:
   - name: letter
     description: A formal written correspondence with sender/recipient addresses, date, salutation, body, and closing signature
@@ -308,8 +309,8 @@ classes:
         description: Additional notes or remarks about the document. Look for sections labeled 'notes', 'remarks', or 'comments'.
 classification:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   top_p: '0.1'
   max_tokens: '4096'
   top_k: '5'
@@ -450,8 +451,8 @@ classification:
   classificationMethod: textbasedHolisticClassification
 extraction:
   image:
-    target_width: '951'
-    target_height: '1268'
+    target_width: ''
+    target_height: ''
   top_p: '0.1'
   max_tokens: '10000'
   top_k: '5'
@@ -608,8 +609,8 @@ summarization:
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   granular:
     enabled: true
     max_workers: "20"

config_library/pattern-2/few_shot_example_with_multimodal_page_classification/config.yaml

@@ -12,8 +12,9 @@ ocr:
     - name: TABLES
     - name: SIGNATURES
   image:
-    target_width: '951'
-    target_height: '1268'
+    dpi: '150'
+    target_width: ''
+    target_height: ''
 classes:
   - name: letter
     description: >-
@@ -647,8 +648,8 @@ classes:
 
 classification:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   classificationMethod: multimodalPageLevelClassification
   model: us.amazon.nova-pro-v1:0
   temperature: '0.0'
@@ -710,8 +711,8 @@ classification:
     {{"class": "letter"}}
 extraction:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   model: us.amazon.nova-pro-v1:0
   temperature: '0.0'
   top_p: '0.1'
@@ -895,8 +896,8 @@ pricing:
         price: '3.75E-6'
 assessment:
   image:
-    target_height: '1268'
-    target_width: '951'
+    target_height: ''
+    target_width: ''
   granular:
     enabled: true
     max_workers: "20"

config_library/pattern-3/default/config.yaml

@@ -12,8 +12,9 @@ ocr:
     - name: TABLES
     - name: SIGNATURES
   image:
-    target_width: '951'
-    target_height: '1268'
+    dpi: '150'
+    target_width: ''
+    target_height: ''
 classes:
   - name: letter
     description: A formal written correspondence with sender/recipient addresses, date, salutation, body, and closing signature
@@ -310,8 +311,8 @@ classification:
   model: Custom fine tuned UDOP model
 extraction:
   image:
-    target_width: '951'
-    target_height: '1268'
+    target_width: ''
+    target_height: ''
   top_p: '0.1'
   max_tokens: '10000'
   top_k: '5'
@@ -468,8 +469,8 @@ summarization:
     You are a document summarization expert who can analyze and summarize documents from various domains including medical, financial, legal, and general business documents. Your task is to create a summary that captures the key information, main points, and important details from the document. Your output must be in valid JSON format. \nSummarization Style: Balanced\\nCreate a balanced summary that provides a moderate level of detail. Include the main points and key supporting information, while maintaining the document's overall structure. Aim for a comprehensive yet concise summary.\n Your output MUST be in valid JSON format with markdown content. You MUST strictly adhere to the output format specified in the instructions.
 assessment:
   image:
-    target_width: '951'
-    target_height: '1268'
+    target_width: ''
+    target_height: ''
   granular:
     enabled: true
     max_workers: "20"

docs/assessment.md

@@ -536,49 +536,76 @@ StateTaxes[0]:
 
 The assessment service supports configurable image dimensions for optimal confidence evaluation:
 
-### Default Configuration
+### New Default Behavior (Preserves Original Resolution)
+
+**Important Change**: Empty strings or unspecified image dimensions now preserve the original document resolution for maximum assessment accuracy:
 
 ```yaml
 assessment:
   model: "anthropic.claude-3-5-sonnet-20241022-v2:0"
-  # Image processing settings
+  # Image processing settings - preserves original resolution
   image:
-    target_width: 951    # Default width in pixels
-    target_height: 1268  # Default height in pixels
+    target_width: ""     # Empty string = no resizing (recommended)
+    target_height: ""    # Empty string = no resizing (recommended)
 ```
 
 ### Custom Image Dimensions
 
-Configure image dimensions based on assessment requirements:
+Configure specific dimensions when performance optimization is needed:
 
 ```yaml
-# For detailed visual assessment
+# For detailed visual assessment with controlled dimensions
 assessment:
   image:
-    target_width: 1200
-    target_height: 1600
+    target_width: "1200"   # Resize to 1200 pixels wide
+    target_height: "1600"  # Resize to 1600 pixels tall
 
 # For standard confidence evaluation
 assessment:
   image:
-    target_width: 800
-    target_height: 1000
+    target_width: "800"    # Smaller for faster processing
+    target_height: "1000"  # Maintains good quality
 ```
 
 ### Image Resizing Features for Assessment
 
-- **Aspect Ratio Preservation**: Images maintain proportions for accurate visual analysis
+- **Original Resolution Preservation**: Empty strings preserve full document resolution for maximum assessment accuracy
+- **Aspect Ratio Preservation**: Images maintain proportions for accurate visual analysis when dimensions are specified
 - **Smart Scaling**: Only downsizes when necessary to preserve visual detail
 - **High-Quality Resampling**: Better image quality for confidence assessment
-- **Performance Optimization**: Optimized images reduce assessment processing time
+- **Performance Optimization**: Configurable dimensions allow balancing accuracy vs. speed
 
 ### Configuration Benefits for Assessment
 
-- **Enhanced Visual Analysis**: Appropriate resolution improves confidence evaluation accuracy
+- **Maximum Assessment Accuracy**: Empty strings preserve full document resolution for best confidence evaluation
+- **Enhanced Visual Analysis**: Original resolution improves confidence evaluation accuracy
 - **Better OCR Verification**: Higher quality images help verify extraction results against visual content
 - **Improved Confidence Scoring**: Better image quality leads to more accurate confidence assessments
 - **Service-Specific Tuning**: Optimize image dimensions for different assessment complexity levels
-- **Resource Optimization**: Balance assessment quality and processing costs
+- **Resource Optimization**: Choose between accuracy (original resolution) and performance (smaller dimensions)
+
+### Migration from Previous Versions
+
+**Previous Behavior**: Empty strings defaulted to 951x1268 pixel resizing
+**New Behavior**: Empty strings preserve original image resolution
+
+If you were relying on the previous default resizing behavior, explicitly set dimensions:
+
+```yaml
+# To maintain previous default behavior
+assessment:
+  image:
+    target_width: "951"
+    target_height: "1268"
+```
+
+### Best Practices for Assessment
+
+1. **Use Empty Strings for High Accuracy**: For critical confidence assessment, use empty strings to preserve original resolution
+2. **Consider Assessment Complexity**: Complex documents with fine details benefit from higher resolution
+3. **Test Assessment Quality**: Evaluate confidence assessment accuracy with your specific document types
+4. **Monitor Resource Usage**: Higher resolution images consume more memory and processing time
+5. **Balance Accuracy vs Performance**: Choose appropriate settings based on your assessment requirements and processing volume
 
 ## Granular Assessment
 

docs/classification.md

@@ -341,49 +341,75 @@ For comprehensive details on configuring few-shot examples, including multimodal
 
 The classification service supports configurable image dimensions for optimal performance and quality:
 
-### Default Configuration
+### New Default Behavior (Preserves Original Resolution)
+
+**Important Change**: Empty strings or unspecified image dimensions now preserve the original document resolution for maximum classification accuracy:
 
 ```yaml
 classification:
   model: us.amazon.nova-pro-v1:0
-  # Image processing settings
+  # Image processing settings - preserves original resolution
   image:
-    target_width: 951    # Default width in pixels
-    target_height: 1268  # Default height in pixels
+    target_width: ""     # Empty string = no resizing (recommended)
+    target_height: ""    # Empty string = no resizing (recommended)
 ```
 
 ### Custom Image Dimensions
 
-Configure image dimensions based on your specific requirements:
+Configure specific dimensions when performance optimization is needed:
 
 ```yaml
-# For high-accuracy classification
+# For high-accuracy classification with controlled dimensions
 classification:
   image:
-    target_width: 1200
-    target_height: 1600
+    target_width: "1200"   # Resize to 1200 pixels wide
+    target_height: "1600"  # Resize to 1600 pixels tall
 
 # For fast processing with lower resolution
 classification:
   image:
-    target_width: 600
-    target_height: 800
+    target_width: "600"    # Smaller for faster processing
+    target_height: "800"   # Maintains reasonable quality
 ```
 
 ### Image Resizing Features
 
-- **Aspect Ratio Preservation**: Images are resized proportionally without distortion
+- **Original Resolution Preservation**: Empty strings preserve full document resolution for maximum accuracy
+- **Aspect Ratio Preservation**: Images are resized proportionally without distortion when dimensions are specified
 - **Smart Scaling**: Only downsizes images when necessary (scale factor < 1.0)
 - **High-Quality Resampling**: Better visual quality after resizing
-- **Performance Optimization**: Smaller, optimized images process faster with lower memory usage
+- **Performance Optimization**: Configurable dimensions allow balancing accuracy vs. speed
 
 ### Configuration Benefits
 
+- **Maximum Classification Accuracy**: Empty strings preserve full document resolution for best results
 - **Service-Specific Tuning**: Each service can use optimal image dimensions
 - **Runtime Configuration**: No code changes needed to adjust image processing
-- **Backward Compatibility**: Default values maintain existing behavior
-- **Memory Optimization**: Configurable dimensions allow memory optimization
-- **Better Resource Utilization**: Service-specific sizing reduces unnecessary processing
+- **Backward Compatibility**: Existing numeric values continue to work as before
+- **Memory Optimization**: Configurable dimensions allow resource optimization
+- **Better Resource Utilization**: Choose between accuracy (original resolution) and performance (smaller dimensions)
+
+### Migration from Previous Versions
+
+**Previous Behavior**: Empty strings defaulted to 951x1268 pixel resizing
+**New Behavior**: Empty strings preserve original image resolution
+
+If you were relying on the previous default resizing behavior, explicitly set dimensions:
+
+```yaml
+# To maintain previous default behavior
+classification:
+  image:
+    target_width: "951"
+    target_height: "1268"
+```
+
+### Best Practices for Classification
+
+1. **Use Empty Strings for High Accuracy**: For critical document classification, use empty strings to preserve original resolution
+2. **Consider Document Types**: Complex layouts benefit from higher resolution, simple text documents may work well with smaller dimensions
+3. **Test Performance Impact**: Higher resolution images provide better accuracy but consume more resources
+4. **Monitor Processing Time**: Balance classification accuracy with processing speed based on your requirements
 
 ## JSON and YAML Output Support