feat: Add Gemini API integration #650

devin-ai-integration · 2025-01-17T02:33:41Z

Fixed method binding issues in GeminiProvider and improved error handling.

Changes:

Use module-level storage for original method
Fix provider initialization in tracker.py
Add graceful error handling for missing API key

Tested:

Ran test_notebook.py successfully
Verified LLM event tracking (2 LLMs recorded)
Session link: https://app.agentops.ai/drilldown?session_id=e0a07af7-c4dd-48d4-b82f-9703c98dff20

Additional Changes:

Add _extract_token_counts helper method for consistent token handling
Make error handling consistent with OpenAI provider
Remove redundant session checks
Improve error message formatting
Add comprehensive documentation
Set "gemini-1.5-flash" as default model value
Move API key configuration to provider level
Add comprehensive test coverage for error handling

Documentation Updates:

Enhanced README.md in examples/gemini_examples/ with:
- Detailed setup instructions
- Code examples for sync and streaming
- Features and notes sections
- Links to additional resources
Updated docs/v1/integrations/gemini.mdx with:
- Automatic provider detection
- Environment setup guide
- Code samples and examples
- Feature documentation
Updated example notebook with auto-initialization
- Removed manual provider.override() calls
- Added graceful API key handling
- Improved error messages

Latest Test Results:

All tests passing
LLM events properly tracked
Session link: https://app.agentops.ai/drilldown?session_id=eac38a27-5897-410f-9bc6-7d8f74497f28

Link to Devin run: https://app.devin.ai/sessions/bf5d3093cfb84eac8a046d4c2f6f992e

- Add GeminiProvider class for tracking Gemini API calls - Support both sync and streaming modes - Track prompts, completions, and token usage - Add test script demonstrating usage Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

devin-ai-integration · 2025-01-17T02:33:44Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
Look at CI failures and help fix them

⚙️ Control Options:

Disable automatic comment and CI monitoring

codecov · 2025-01-17T02:34:48Z

Codecov Report

Attention: Patch coverage is 16.52174% with 96 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
agentops/llms/providers/gemini.py	16.82%	89 Missing ⚠️
agentops/llms/tracker.py	12.50%	7 Missing ⚠️

📢 Thoughts on this report? Let us know!

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

…ing in Gemini provider Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

…orting Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

…improve test coverage Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

…dling and event recording Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

areibman · 2025-01-17T10:14:56Z

Devin did good I think, check it out @the-praxs

dot-agi

Overall the integration was done well. Such small integrations can be done with Devin but the maintenance will be the real challenge.

Comments are left to improve the code structure and efficiency. The notebooks should be modified to follow the same structure as the other notebooks.

Docs are absent. They need to be added for a complete integration.

Sometime later would like to have ActionEvent and ToolEvent events also if they are supported.

agentops/llms/tracker.py

agentops/llms/providers/gemini.py

- Add _extract_token_counts helper method - Make error handling consistent with OpenAI provider - Remove redundant session checks - Improve error message formatting - Add comprehensive documentation Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

dot-agi · 2025-01-18T18:56:01Z

I would like to state that this was mergeable because I fixed it.

devin-ai-integration bot and others added 3 commits January 17, 2025 02:25

feat: Add Gemini API integration

1939b6d

- Add GeminiProvider class for tracking Gemini API calls - Support both sync and streaming modes - Track prompts, completions, and token usage - Add test script demonstrating usage Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Pass session correctly to track LLM events in Gemini provider

9e4f471

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

feat: Add Gemini integration with example notebook

b95fe6e

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

devin-ai-integration bot and others added 25 commits January 17, 2025 02:38

fix: Add null checks and improve test coverage for Gemini provider

72e985a

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Add blank lines between test functions

6df9b7e

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Improve test coverage for Gemini provider

200dcf1

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Fix formatting in test_gemini.py

cd31098

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add comprehensive test coverage for edge cases and error handling

fef63a9

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add graceful API key handling and skip tests when key is missing

10900f5

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Fix formatting issues in test files

4b96b0f

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Remove trailing whitespace in test_gemini.py

062f82d

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add coverage for error handling, edge cases, and argument handl…

d418202

…ing in Gemini provider Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add streaming exception handling test coverage

a9cea74

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Apply ruff auto-formatting to test_gemini.py

11c7343

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Fix type errors and improve test coverage for Gemini provider

4f0b0fe

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add comprehensive error handling test coverage for Gemini provider

1a6e1ca

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Apply ruff-format fixes to test_gemini.py

9efc0f1

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Configure Gemini API key before model initialization

071a610

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Update GeminiProvider to properly handle instance methods

970c318

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Use provider instance in closure for proper method binding

18143b5

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Use class-level storage for original method

a27b2e4

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Use module-level storage for original method

aed3a1b

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Apply ruff-format fixes to Gemini integration

8297371

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Move Gemini tests to unit test directory for proper coverage rep…

9c9af3a

…orting Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Update Gemini provider to properly handle prompt extraction and …

bff477c

…improve test coverage Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add comprehensive test coverage for Gemini provider session han…

f8fd56d

…dling and event recording Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

style: Apply ruff-format fixes to test files

59db821

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Pass LlmTracker client to GeminiProvider constructor

f163e23

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

remove extra files

6d7ee0f

areibman requested a review from dot-agi January 17, 2025 10:14

dot-agi requested changes Jan 17, 2025

View reviewed changes

devin-ai-integration bot and others added 13 commits January 17, 2025 11:27

chore: Clean up test files and merge remote changes

54a9d36

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

test: Add comprehensive test coverage for Gemini provider

c845a34

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

fix: Set None as default values and improve test coverage

973e59f

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

build: Add google-generativeai as test dependency

481a8d7

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

docs: Update examples and README for Gemini integration

0871398

Co-Authored-By: Alex Reibman <meta.alex.r@gmail.com>

add gemini logo image

cddab5b

add gemini to examples

681cd18

add gemini to docs

9e8e85e

refactor handle_response method

e75fa84

cleanup gemini tracking code

86dec80

delete unit test for gemini

3384b2d

rename and clean gemini example notebook

392677a

dot-agi requested a review from areibman January 18, 2025 18:52

dot-agi added 2 commits January 19, 2025 00:22

ruff

38e2621

update docs

9e3393d

dot-agi self-requested a review January 18, 2025 18:54

dot-agi enabled auto-merge (squash) January 18, 2025 18:54

dot-agi disabled auto-merge January 18, 2025 18:54

dot-agi enabled auto-merge (squash) January 18, 2025 18:55

areibman approved these changes Jan 18, 2025

View reviewed changes

dot-agi approved these changes Jan 18, 2025

View reviewed changes

dot-agi merged commit 543b180 into main Jan 18, 2025
8 of 10 checks passed

dot-agi deleted the devin/1737080675-gemini-integration branch January 18, 2025 18:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add Gemini API integration #650

feat: Add Gemini API integration #650

Uh oh!

devin-ai-integration bot commented Jan 17, 2025 •

edited

Loading

Uh oh!

devin-ai-integration bot commented Jan 17, 2025

Uh oh!

codecov bot commented Jan 17, 2025 •

edited

Loading

Uh oh!

areibman commented Jan 17, 2025

Uh oh!

dot-agi left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dot-agi commented Jan 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Add Gemini API integration #650

feat: Add Gemini API integration #650

Uh oh!

Conversation

devin-ai-integration bot commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devin-ai-integration bot commented Jan 17, 2025

🤖 Devin AI Engineer

Uh oh!

codecov bot commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

areibman commented Jan 17, 2025

Uh oh!

dot-agi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dot-agi commented Jan 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devin-ai-integration bot commented Jan 17, 2025 •

edited

Loading

codecov bot commented Jan 17, 2025 •

edited

Loading

dot-agi left a comment •

edited

Loading