Update: [AEA-6083] - Llama Prompt Engineering

[kieran-wilkinson-4]

Summary

Improved bot responses

Details

• Moved from "Amazon Nova" to "Meta Llama"
  • Nova doesn't handle 0.0 temperature (min 0.0001) which still creates significant variation in responses
  • Nova's prompt engineering was very different to Anthropic's Claude
    • Llama is a lot more similar, meaning the prompts can be much easier switched over
• Fixed regex for "Markdown" -> (Slacks) "Mrkdwn"
  • Fixed tests for regex
• Updated System Prompt to be shorter and stricter
  • Longer prompts weaken directions, leading to mistakes in responses or the bot ignoring directions altogether
  • Longer prompts also eat up tokens (which we want to minimise)
  • Providing examples strengthens understanding
• Increased logging throughout response generation
  • Inc. Response from Bedrock, parsing, formatting, and Slack's block building

[github-actions]

This PR is linked to a ticket in an NHS Digital JIRA Project. Here's a handy link to the ticket:

AEA-6083

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud