Skip to content

Conversation

@amanjaiswal73892
Copy link
Collaborator

@amanjaiswal73892 amanjaiswal73892 commented Jul 31, 2025

  • Implement LiteLLMModel and LiteLLMAPIMessageBuilder for LiteLLM API interaction.
  • Enhance APIPayload to include reasoning_effort parameter.
  • Introduce get_pricing_litellm function for pricing information retrieval.
  • Create tests for multi-action, single tool call scenarios and force tool call.
  • Added an option to discard extra tool calls by setting use_only_first_toolcall.

LiteLLM can be used with tool use agent as follows:

from agentlab.agents.tool_use_agent import DEFAULT_PROMPT_CONFIG, ToolUseAgentArgs
from agentlab.experiments.study import Study
from agentlab.llm.litellm_api import LiteLLMModelArgs

def get_agent(model_name: str) -> ToolUseAgentArgs:
    return ToolUseAgentArgs(
        model_args=LiteLLMModelArgs(
            model_name=model_name,
            max_new_tokens=2000,
            temperature=None,
            # use_only_first_toolcall=True,  # Set to True for single actions
        ),
        config=DEFAULT_PROMPT_CONFIG,
    )

models = [
    "openai/gpt-4.1",
    "openai/gpt-4.1-mini",
    "openai/gpt-4.1-nano",
    "openai/o3-2025-04-16",
    "anthropic/claude-3-7-sonnet-20250219",
    "anthropic/claude-sonnet-4-20250514",
    ## Add more models to test.
]
agent_args = [get_agent(model) for model in models]


study = Study(agent_args, "miniwob_tiny_test", logging_level_stdout=logging.WARNING)
study.run(
    n_jobs=5,
    parallel_backend="ray",
    strict_reproducibility=False,
    n_relaunch=3,
)

Description by Korbit AI

What change is being made?

Add integration of the LiteLLM API, including the implementation of the LiteLLMModel class for interfacing with the LiteLLM service, updates to the pricing tracker, and accompanying tests to ensure its functionalities.

Why are these changes being made?

Integrating the LiteLLM API allows seamless use of the LiteLLM functionality within our codebase, enhancing model interaction capabilities by providing a wrapper for the OpenAI's API with additional features such as tool calls and pricing integration. This change aims to expand our platform's compatibility with different language models and enhance the ability to choose specific tools or actions during AI processes, advancing the system's adaptability and functionality.

Is this description stale? Ask me to generate a new description by commenting /korbit-generate-pr-description

- Implement LiteLLMModel and LiteLLMAPIMessageBuilder for LiteLLM API interaction.
- Enhance APIPayload to include reasoning_effort parameter.
- Introduce get_pricing_litellm function for pricing information retrieval.
- Create  tests for multi-action, single tool call scenarios and force tool call.
@korbit-ai
Copy link

korbit-ai bot commented Jul 31, 2025

Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment /korbit-review.

Your admin can change your review schedule in the Korbit Console

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants