Enable LoRA auto-detection for Intent/Category Classification

[yossiovadia]

Summary

Following the request from @Xunzhuo in PR #718, this issue tracks the migration of Intent/Category Classification to support LoRA auto-detection.

Background

Currently, PII detection already has LoRA auto-detection (implemented in PR #709), but Intent Classification does not. This creates an inconsistency where some classification features can leverage LoRA models while others cannot.

Current Behavior (BEFORE)

Problem: Intent classification cannot automatically use LoRA models.

How it works:

1. Configuration has a use_modernbert flag that determines which initializer to use
2. System makes a hardcoded choice between two paths:
   • use_modernbert: false → Uses LinearCategoryInitializer (Traditional BERT only)
   • use_modernbert: true → Uses ModernBertCategoryInitializer (ModernBERT only)
3. Neither path can detect or use LoRA models automatically
4. Even if you point model_id to a LoRA model, it will fail because the initializer doesn't understand LoRA

Example:

category_model:
  model_id: "models/lora_intent_classifier_bert-base-uncased_model"
  use_modernbert: false  # Will try Linear BERT, FAIL with LoRA model

Expected Behavior (AFTER)

Solution: Intent classification should auto-detect LoRA models (just like PII does).

How it should work:

1. Single auto-detecting initializer that intelligently routes based on model type
2. Detection happens automatically by checking:
   • LoRA weights in model.safetensors file
   • Presence of lora_config.json
3. Smart fallback chain: LoRA → Traditional BERT → ModernBERT
4. The use_modernbert config flag becomes optional/ignored (backward compatible)
5. Zero configuration needed - just point to model path and system figures it out

Example (same config, now works):

category_model:
  model_id: "models/lora_intent_classifier_bert-base-uncased_model"
  use_modernbert: false  # Ignored - auto-detection finds LoRA and uses it

Implementation Notes

• Follow the same pattern as PII detection (PR fix(647): enable LoRA PII auto-detection with minimal changes #709)
• Update both Go layer (classifier.go) and Rust layer (init.rs, classify.rs)
• Add auto-detection test similar to PII
• Ensure backward compatibility with existing Traditional BERT and ModernBERT models
• LoRA intent models already exist in the models/ directory

Related Work

• PR fix(647): enable LoRA PII auto-detection with minimal changes #709: PII LoRA auto-detection (reference implementation)
• PR fix(api): expose actual PII confidence scores instead of hardcoded 0.9 #718: Fixed PII API confidence scores, comment requesting this migration
• Issue #TBD: Jailbreak Detection LoRA auto-detection (depends on this being merged first)

Success Criteria

• LoRA intent models are automatically detected and used
• Traditional BERT and ModernBERT fallback paths still work
• No configuration changes required for users
• All existing tests continue to pass
• New tests demonstrate LoRA auto-detection working