Merge branch 'feature/chat-with-document' into 'develop'

Feature/chat with document

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!232

Author: rstrahan

docs/architecture.md

@@ -41,6 +41,8 @@ The solution uses a modular architecture with nested CloudFormation stacks to su
 - Pattern-specific optimizations and configurations
 - Optional features that can be enabled across all patterns:
   - Document summarization (controlled by `IsSummarizationEnabled` parameter)
+    - This feature also enables the "Chat with Document" functionality
+    - This feature does not use the Bedrock Knowledge Base but stores a full-text text file in S3
   - Document Knowledge Base (using Amazon Bedrock)
   - Automated accuracy evaluation against baseline data
 

docs/web-ui.md

@@ -22,6 +22,7 @@ The solution includes a responsive web-based user interface built with React tha
 - **Confidence threshold configuration** for HITL (Human-in-the-Loop) triggering through the Assessment & HITL Configuration section
 - Document upload from local computer
 - Knowledge base querying for document collections
+- "Chat with document" from the detailed view of the document
 - **Document Process Flow visualization** for detailed workflow execution monitoring and troubleshooting
 - **Document Analytics** for querying and visualizing processed document data
 
@@ -97,6 +98,17 @@ The Document Process Flow visualization is particularly useful for troubleshooti
 - Analyze execution times to identify performance bottlenecks
 - Inspect the input and output of each step to verify data transformation
 
+## Chat with Document
+
+The "Chat with Document" feature is available at the bottom of the Document Detail view. This feature uses the same model that's configured to do the summarization to provide a RAG interface to the document that's the details are displayed for. No other document is taken in to account except the document you're viewing the details of. Note that this feature will only work after the document status is marked as complete.
+
+Your chat history will be saved as you continue your chat but if you leave the document details screen, your chat history is erased. This feature uses prompt caching for the document contents for repeated chat requests for each document.
+
+### How to Use
+
+1. Navigate to a document's detail page and scroll to the bottom
+2. In the text area, type in your question and you'll see an answer pop up after the document is analyzed with the model that's configured for summarization
+
 ## Authentication Features
 
 The web UI uses Amazon Cognito for secure user authentication and authorization:

lib/idp_common_pkg/idp_common/summarization/service.py

@@ -575,6 +575,16 @@ def process_document(
                     content_type="application/json",
                 )
 
+                # Store the full text for chat
+                all_text = self._get_all_text(document)
+                fulltext_key = f"{document.input_key}/summary/fulltext.txt"
+                s3.write_content(
+                    content=all_text,
+                    bucket=output_bucket,
+                    key=fulltext_key,
+                    content_type="text/plain",
+                )
+
                 # Create and store the combined markdown summary
                 md_key = f"{document.input_key}/summary/summary.md"
 
@@ -639,6 +649,30 @@ def process_document(
 
         return document
 
+    def _get_all_text(self, document: Document) -> str:
+        """
+        Retrieve all text content from a document's pages.
+
+        Args:
+            document: Document object to process
+
+        Returns:
+            str: Combined text content from all pages
+        """
+        all_text = ""
+        for page_id, page in sorted(document.pages.items()):
+            if page.parsed_text_uri:
+                try:
+                    page_text = s3.get_text_content(page.parsed_text_uri)
+                    all_text += f"<page-number>{page_id}</page-number>\n{page_text}\n\n"
+                except Exception as e:
+                    logger.warning(
+                        f"Failed to load text content from {page.parsed_text_uri}: {e}"
+                    )
+                    # Continue with other pages
+
+        return all_text
+
     def _process_document_as_whole(
         self, document: Document, store_results: bool = True
     ) -> Document:
@@ -659,19 +693,7 @@ def _process_document_as_whole(
             start_time = time.time()
 
             # Combine text from all pages
-            all_text = ""
-            for page_id, page in sorted(document.pages.items()):
-                if page.parsed_text_uri:
-                    try:
-                        page_text = s3.get_text_content(page.parsed_text_uri)
-                        all_text += (
-                            f"<page-number>{page_id}</page-number>\n{page_text}\n\n"
-                        )
-                    except Exception as e:
-                        logger.warning(
-                            f"Failed to load text content from {page.parsed_text_uri}: {e}"
-                        )
-                        # Continue with other pages
+            all_text = self._get_all_text(document)
 
             if not all_text:
                 logger.warning("No text content found in document pages")
@@ -708,7 +730,7 @@ def _process_document_as_whole(
                     content_type="application/json",
                 )
 
-                # Store the raw text
+                # Store the full text for chat
                 fulltext_key = f"{document.input_key}/summary/fulltext.txt"
                 s3.write_content(
                     content=all_text,

lib/idp_common_pkg/tests/unit/summarization/test_summarization_service.py

@@ -450,8 +450,8 @@ def test_process_document(
             # Verify executor was used to process sections in parallel
             assert mock_executor_instance.submit.call_count == 2
 
-            # Verify write_content was called for combined results
-            assert mock_write_content.call_count == 2
+            # Verify write_content was called for combined results (JSON, fulltext, and markdown)
+            assert mock_write_content.call_count == 3
 
             # Verify document has summarization_result
             assert result.summarization_result is not None

src/api/schema.graphql

@@ -203,7 +203,7 @@ type Query @aws_cognito_user_pools @aws_iam {
   getFileContents(s3Uri: String!): FileContentsResponse
   getConfiguration: ConfigurationResponse
   queryKnowledgeBase(input: String!, sessionId: String): String
-  chatWithDocument(s3Uri: String!, prompt: String!): String
+  chatWithDocument(s3Uri: String!, prompt: String!, history: AWSJSON!, modelId: String!): String
   getStepFunctionExecution(executionArn: String!): StepFunctionExecutionResponse
   submitAnalyticsQuery(query: String!): AnalyticsJob @aws_cognito_user_pools
   getAnalyticsJobStatus(jobId: ID!): AnalyticsJob @aws_cognito_user_pools

src/lambda/chat_with_document_resolver/index.py

@@ -0,0 +1,152 @@
+import json
+import boto3
+import logging
+import botocore
+import html
+import mimetypes
+import base64
+import hashlib
+import os
+from urllib.parse import urlparse
+from botocore.exceptions import ClientError
+
+# Set up logging
+logger = logging.getLogger()
+logger.setLevel(os.environ.get("LOG_LEVEL", "INFO"))
+# Get LOG_LEVEL from environment variable with INFO as default
+
+def get_summarization_model():
+    """Get the summarization model from configuration table"""
+    try:
+        dynamodb = boto3.resource('dynamodb')
+        config_table = dynamodb.Table(os.environ['CONFIGURATION_TABLE_NAME'])
+        
+        # Query for the Default configuration
+        response = config_table.get_item(
+            Key={'Configuration': 'Default'}
+        )
+        
+        if 'Item' in response:
+            config_data = response['Item']
+            # Extract summarization model from the configuration
+            if 'summarization' in config_data and 'model' in config_data['summarization']:
+                return config_data['summarization']['model']
+        
+        # Fallback to a default model if not found in config
+        return 'us.amazon.nova-pro-v1:0'
+        
+    except Exception as e:
+        logger.error(f"Error getting summarization model from config: {str(e)}")
+        return 'us.amazon.nova-pro-v1:0'  # Fallback default
+
+def handler(event, context):
+    response_data = {}
+
+    try:
+        # logger.info(f"Received event: {json.dumps(event)}")
+
+        objectKey = event['arguments']['s3Uri']
+        prompt = event['arguments']['prompt']
+        history = event['arguments']['history']
+
+        full_prompt = "You are an assistant that's responsible for getting details from document text attached here based on questions from the user.\n\n"
+        full_prompt += "If you don't know the answer, just say that you don't know. Don't try to make up an answer.\n\n"
+        full_prompt += "Additionally, use the user and assistant responses in the following JSON object to see what's been asked and what the resposes were in the past.\n\n"
+        full_prompt += "The JSON object is: " + json.dumps(history) + ".\n\n"
+        full_prompt += "The user's question is: " + prompt
+
+        # this feature is not enabled until the model can be selected on the chat screen
+        # selectedModelId = event['arguments']['modelId']
+        selectedModelId = get_summarization_model()
+
+        logger.info(f"Processing S3 URI: {objectKey}")
+
+        output_bucket = os.environ['OUTPUT_BUCKET']
+
+        bedrock_runtime = boto3.client('bedrock-runtime', region_name='us-west-2')
+
+        # Call Bedrock Runtime to get Python code based on the prompt
+        if (len(objectKey)):
+            encoded_string = objectKey.encode()
+            md5_hash = hashlib.md5(encoded_string, usedforsecurity=False)
+            hex_representation = md5_hash.hexdigest()
+
+            # full text key
+            fulltext_key = objectKey + '/summary/fulltext.txt'
+
+            logger.info(f"Output Bucket: {output_bucket}")
+            logger.info(f"Full Text Key: {fulltext_key}")
+
+            # read full contents of the object as text
+            s3 = boto3.client('s3')
+            response = s3.get_object(Bucket=output_bucket, Key=fulltext_key)
+            content_str = response['Body'].read().decode('utf-8')
+
+            message = [
+                {
+                    "role":"user",
+                    "content": [
+                        {
+                            "text": content_str
+                        },
+                        {
+                           "cachePoint" : {
+                                'type': 'default'
+                            }
+                        }
+                    ]
+                },
+                {
+                    "role":"user",
+                    "content": [
+                        {
+                            "text": full_prompt
+                        }
+                    ]
+                }
+            ]
+
+            # print('invoking model converse')
+
+            response = bedrock_runtime.converse(
+                modelId=selectedModelId,
+                messages=message
+            )
+
+            token_usage = response['usage']
+            # print(f"Input tokens:  {token_usage['inputTokens']}")
+            # print(f"Output tokens:  {token_usage['outputTokens']}")
+            # print(f"Total tokens:  {token_usage['totalTokens']}")
+            # print(f"cacheReadInputTokens:  {token_usage['cacheReadInputTokens']}")
+            # print(f"cacheWriteInputTokens:  {token_usage['cacheWriteInputTokens']}")
+            # print(f"Stop reason: {response['stopReason']}")
+
+            output_message = response['output']['message']
+
+            model_response_text = ''
+            for content in output_message['content']:
+                model_response_text += content['text']
+
+            # print output_message
+
+            chat_response = {"cr" : output_message }
+            return json.dumps(chat_response)
+
+
+    except ClientError as e:
+        error_code = e.response['Error']['Code']
+        error_message = e.response['Error']['Message']
+        logger.error(f"S3 ClientError: {error_code} - {error_message}")
+        
+        if error_code == 'NoSuchKey':
+            raise Exception(f"File not found: {objectKey}")
+        elif error_code == 'NoSuchBucket':
+            raise Exception(f"Bucket not found: {output_bucket}")
+        else:
+            raise Exception(error_message)
+            
+    except Exception as e:
+        logger.error(f"Unexpected error: {str(e)}")
+        raise Exception(f"Error fetching file: {str(e)}")
+    
+    return response_data

src/lambda/chat_with_document_resolver/requirements.txt

@@ -0,0 +1 @@
+boto3>=1.38.45

src/ui/package-lock.json

@@ -23,6 +23,34 @@
     white-space: pre-wrap;
 }
 
+.chat-assistant.error {
+    background-color: rgb(245, 178, 178);
+}
+
 .chat-composer-container {
     margin-top: 10px;
+}
+
+.time {
+    font-size: 9px;
+}
+
+/* HTML: <div class="loader"></div> */
+.loader {
+  width: 60px;
+  aspect-ratio: 2;
+  --_g: no-repeat radial-gradient(circle closest-side,#000 90%,#0000);
+  background: 
+    var(--_g) 0%   50%,
+    var(--_g) 50%  50%,
+    var(--_g) 100% 50%;
+  background-size: calc(100%/5) 60%;
+  animation: l3 1s infinite linear;
+  margin: 10px 5px;
+}
+@keyframes l3 {
+    20%{background-position:0%   0%, 50%  50%,100%  50%}
+    40%{background-position:0% 100%, 50%   0%,100%  50%}
+    60%{background-position:0%  50%, 50% 100%,100%   0%}
+    80%{background-position:0%  50%, 50%  50%,100% 100%}
 }