fix(prompts): evict prompt cache on NotFound error (#1442)

[hassiebp]

Important

Evict prompt cache entries on NotFoundError during refresh in client.py and add corresponding test.

• Behavior:
  • In client.py, fetch_prompts now evicts cache entry on NotFoundError by calling delete() on PromptCache.
  • Logs a warning when a prompt is not found and evicted from cache.
• Cache Management:
  • Adds delete() method to PromptCache in prompt_cache.py to remove cache entries by key.
• Testing:
  • Adds test_evict_prompt_cache_entry_when_refresh_returns_not_found in test_prompt.py to verify cache eviction on NotFoundError.

^{This description was created by for 24e7924. You can customize this summary. It will automatically update as commits are pushed.}

Disclaimer: Experimental PR review

Greptile Overview

Greptile Summary

Fixes a cache staleness issue where deleted prompts remain cached indefinitely by evicting cache entries when the API returns a NotFoundError.

Changes:

• Added NotFoundError exception handling in _fetch_prompt_and_update_cache() to detect when prompts are deleted from the server
• Implemented delete() method in PromptCache class to safely remove cache entries
• Added comprehensive test coverage for the cache eviction behavior when prompts return NotFoundError during background refresh
• Updated test fixture to properly initialize _resources with a real PromptCache instance

Impact:
This ensures that when a prompt is deleted on the server, subsequent calls after cache expiry will properly receive a NotFoundError and fall back to the fallback prompt (if provided) instead of serving stale cached data indefinitely.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The implementation is clean, well-tested, and follows established patterns. The fix properly handles a specific edge case (deleted prompts) without affecting existing behavior. The delete() method uses safe pop() with a default value, preventing KeyError. The new exception handler is placed before the generic exception handler, ensuring correct catch order. The test comprehensively validates the new behavior including background refresh scenarios.
• No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
langfuse/_client/client.py	5/5	Added NotFoundError handling to evict stale cache entries when prompts are deleted, with proper import and error handling
langfuse/_utils/prompt_cache.py	5/5	Added delete() method to remove cache entries by key using safe pop() operation
tests/test_prompt.py	4/5	Added comprehensive test for cache eviction on NotFoundError and updated fixture to properly initialize _resources

Sequence Diagram

sequenceDiagram
    participant Client as Client Code
    participant Langfuse as Langfuse.get_prompt()
    participant Cache as PromptCache
    participant Fetch as _fetch_prompt_and_update_cache()
    participant API as API.prompts.get()

    Client->>Langfuse: get_prompt(name, cache_ttl)
    Langfuse->>Cache: get(cache_key)
    
    alt Cache Hit (Expired)
        Cache-->>Langfuse: cached_prompt (expired)
        Langfuse->>Cache: add_refresh_prompt_task()
        Note over Cache: Background refresh triggered
        Langfuse-->>Client: return stale prompt
        
        Cache->>Fetch: refresh in background
        Fetch->>API: prompts.get()
        
        alt Prompt Deleted (NotFoundError)
            API-->>Fetch: NotFoundError
            Note over Fetch: NEW: Catch NotFoundError
            Fetch->>Cache: delete(cache_key)
            Note over Cache: Cache entry evicted
            Fetch-->>Cache: raise NotFoundError
        else Success
            API-->>Fetch: prompt data
            Fetch->>Cache: set(cache_key, prompt)
        end
    else Cache Miss
        Cache-->>Langfuse: None
        Langfuse->>Fetch: fetch directly
        Fetch->>API: prompts.get()
        
        alt NotFoundError
            API-->>Fetch: NotFoundError
            Fetch->>Cache: delete(cache_key)
            Fetch-->>Langfuse: raise NotFoundError
            Langfuse-->>Client: fallback or error
        else Success
            API-->>Fetch: prompt data
            Fetch->>Cache: set(cache_key, prompt)
            Fetch-->>Langfuse: prompt
            Langfuse-->>Client: return prompt
        end
    end

Loading

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}