Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,35 @@
# Path to your iFlow credential file (e.g., ~/.iflow/oauth_creds.json).
#IFLOW_OAUTH_1=""

# --- Kiro CLI / Kiro IDE ---
# Multiple credential sources are supported (in priority order):
#
# Option 1: Direct refresh token (simplest for Docker/stateless deployments)
#KIRO_REFRESH_TOKEN="your_kiro_refresh_token"
# or
#REFRESH_TOKEN="your_kiro_refresh_token"
#
# Option 2: JSON credentials file (from Kiro IDE)
# Path to kiro-auth-token.json from Kiro IDE
#KIRO_CREDS_FILE="~/.aws/sso/cache/kiro-auth-token.json"
#
# Option 3: SQLite database (from kiro-cli)
# Path to kiro-cli SQLite database
#KIRO_CLI_DB_FILE=""
#
# If none are set, the proxy will auto-detect from default paths:
# JSON: ~/.aws/sso/cache/kiro-auth-token.json
# SQLite (macOS): ~/Library/Application Support/kiro-cli/data.sqlite3
# SQLite (macOS): ~/Library/Application Support/amazon-q/data.sqlite3
# SQLite (Linux): ~/.local/share/kiro-cli/data.sqlite3
# SQLite (Linux): ~/.local/share/amazon-q/data.sqlite3

# Optional profile ARN (only needed for Kiro Desktop auth, not for AWS SSO)
#PROFILE_ARN="arn:aws:codewhisperer:us-east-1:..."

# Optional override for Kiro API region (default: us-east-1)
#KIRO_REGION="us-east-1"


# ------------------------------------------------------------------------------
# | [ADVANCED] Provider-Specific Settings |
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -131,4 +131,4 @@ cache/
*.env

oauth_creds/

reference/
20 changes: 10 additions & 10 deletions Deployment guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Before deploying, try the proxy locally to ensure your keys work. This uses a pr
2. Download the latest release ZIP file (e.g., for Windows).
3. Unzip the file.
4. Double-click `setup_env.bat`. A window will open—follow the prompts to add your PROXY_API_KEY (a strong secret you create) and provider keys. Use the [LiteLLM Providers Documentation](https://docs.litellm.ai/docs/providers) for guidance on key formats (e.g., `GEMINI_API_KEY_1="your-key"`).
5. Double-click `proxy_app.exe` to start the proxy. It runs at `http://127.0.0.1:8000`—visit in a browser to confirm "API Key Proxy is running".
5. Double-click `proxy_app.exe` to start the proxy. It runs at `http://127.0.0.1:7777`—visit in a browser to confirm "API Key Proxy is running".
6. Test with curl (replace with your PROXY_API_KEY):

```
curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
curl -X POST http://127.0.0.1:7777/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer your-proxy-key" -d '{"model": "gemini/gemini-2.5-flash", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
```

- Expected: A JSON response with the answer (e.g., "Paris").
Expand Down Expand Up @@ -220,7 +220,7 @@ docker compose ps
docker compose logs -f

# Test the endpoint
curl http://localhost:8000/
curl http://localhost:7777/
```

### Manual Docker Run
Expand All @@ -236,7 +236,7 @@ touch key_usage.json
docker run -d \
--name llm-api-proxy \
--restart unless-stopped \
-p 8000:8000 \
-p 7777:7777 \
-v $(pwd)/.env:/app/.env:ro \
-v $(pwd)/oauth_creds:/app/oauth_creds \
-v $(pwd)/logs:/app/logs \
Expand Down Expand Up @@ -378,7 +378,7 @@ The image is built for both `linux/amd64` and `linux/arm64` architectures, so it
| Container exits immediately | Check logs: `docker compose logs` — likely missing `.env` or invalid config |
| Permission denied on volumes | Ensure directories exist and have correct permissions: `mkdir -p oauth_creds logs && chmod 755 oauth_creds logs` |
| OAuth credentials not loading | Verify `oauth_creds/` is mounted and contains valid JSON files, or check environment variables are set |
| Port already in use | Change the port mapping: `-p 9000:8000` or edit `docker-compose.yml` |
| Port already in use | Change the port mapping: `-p 9000:7777` or edit `docker-compose.yml` |
| Image not updating | Force pull: `docker compose pull && docker compose up -d` |

---
Expand Down Expand Up @@ -488,7 +488,7 @@ nano .env
# Also add your PROXY_API_KEY and other provider keys

# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 7777
```

**Method B: Upload Credential Files**
Expand All @@ -501,7 +501,7 @@ scp -r oauth_creds/ user@your-vps-ip:/path/to/LLM-API-Key-Proxy/
ls -la oauth_creds/

# Start the proxy
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 7777
```

> **Note**: Environment variables are preferred for production deployments (more secure, easier to manage, works with container orchestration).
Expand Down Expand Up @@ -572,7 +572,7 @@ Still in the credential tool:
# Tunnels are no longer needed

# Start the proxy on VPS (in a screen/tmux session or as a service)
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 7777
```

---
Expand Down Expand Up @@ -650,7 +650,7 @@ OAuth callback ports should **never** be publicly exposed:
# sudo ufw allow 11451/tcp

# ✅ Only open your proxy API port
sudo ufw allow 8000/tcp
sudo ufw allow 7777/tcp

# Check firewall status
sudo ufw status
Expand All @@ -677,7 +677,7 @@ Type=simple
User=your-username
WorkingDirectory=/path/to/LLM-API-Key-Proxy
Environment="PATH=/path/to/python/bin"
ExecStart=/path/to/python/bin/uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
ExecStart=/path/to/python/bin/uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 7777
Restart=always
RestartSec=10

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,12 @@ COPY src/ ./src/
RUN mkdir -p logs oauth_creds

# Expose the default port
EXPOSE 8000
EXPOSE 7777

# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONPATH=/app/src

# Default command - runs proxy with the correct PYTHONPATH
CMD ["python", "src/proxy_app/main.py", "--port", "8000"]
CMD ["python", "src/proxy_app/main.py", "--port", "7777"]
83 changes: 70 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This project consists of two components:
- **One Endpoint, Many Providers** — Configure Gemini, OpenAI, Anthropic, and [any LiteLLM-supported provider](https://docs.litellm.ai/docs/providers) once. Access them all through a single API key
- **Anthropic API Compatible** — Use Claude Code or any Anthropic SDK client with non-Anthropic providers like Gemini, OpenAI, or custom models
- **Built-in Resilience** — Automatic key rotation, failover on errors, rate limit handling, and intelligent cooldowns
- **Exclusive Provider Support** — Includes custom providers not available elsewhere: **Antigravity** (Gemini 3 + Claude Sonnet/Opus 4.5), **Gemini CLI**, **Qwen Code**, and **iFlow**
- **Exclusive Provider Support** — Includes custom providers not available elsewhere: **Antigravity** (Gemini 3 + Claude Sonnet/Opus 4.5), **Gemini CLI**, **Kiro CLI**, **Qwen Code**, and **iFlow**

---

Expand Down Expand Up @@ -49,7 +49,7 @@ chmod +x proxy_app
# Pull and run directly
docker run -d \
--name llm-api-proxy \
-p 8000:8000 \
-p 7777:7777 \
-v $(pwd)/.env:/app/.env:ro \
-v $(pwd)/oauth_creds:/app/oauth_creds \
-v $(pwd)/logs:/app/logs \
Expand Down Expand Up @@ -81,7 +81,7 @@ pip install -r requirements.txt
python src/proxy_app/main.py
```

> **Tip:** Running with command-line arguments (e.g., `--host 0.0.0.0 --port 8000`) bypasses the TUI and starts the proxy directly.
> **Tip:** Running with command-line arguments (e.g., `--host 0.0.0.0 --port 7777`) bypasses the TUI and starts the proxy directly.

---

Expand All @@ -91,7 +91,7 @@ Once the proxy is running, configure your application with these settings:

| Setting | Value |
|---------|-------|
| **Base URL / API Endpoint** | `http://127.0.0.1:8000/v1` |
| **Base URL / API Endpoint** | `http://127.0.0.1:7777/v1` |
| **API Key** | Your `PROXY_API_KEY` |

### Model Format: `provider/model_name`
Expand All @@ -104,6 +104,7 @@ openai/gpt-4o ← OpenAI API
anthropic/claude-3-5-sonnet ← Anthropic API
openrouter/anthropic/claude-3-opus ← OpenRouter
gemini_cli/gemini-2.5-pro ← Gemini CLI (OAuth)
kiro_cli/claude-sonnet-4.5 ← Kiro CLI (Amazon Q/Kiro)
antigravity/gemini-3-pro-preview ← Antigravity (Gemini 3, Claude Opus 4.5)
```

Expand All @@ -116,7 +117,7 @@ antigravity/gemini-3-pro-preview ← Antigravity (Gemini 3, Claude Opus 4.5)
from openai import OpenAI

client = OpenAI(
base_url="http://127.0.0.1:8000/v1",
base_url="http://127.0.0.1:7777/v1",
api_key="your-proxy-api-key"
)

Expand All @@ -133,7 +134,7 @@ print(response.choices[0].message.content)
<summary><b>curl</b></summary>

```bash
curl -X POST http://127.0.0.1:8000/v1/chat/completions \
curl -X POST http://127.0.0.1:7777/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-proxy-api-key" \
-d '{
Expand All @@ -150,7 +151,7 @@ curl -X POST http://127.0.0.1:8000/v1/chat/completions \
1. Go to **API Settings**
2. Select **"Proxy"** or **"Custom OpenAI"** mode
3. Configure:
- **API URL:** `http://127.0.0.1:8000/v1`
- **API URL:** `http://127.0.0.1:7777/v1`
- **API Key:** Your `PROXY_API_KEY`
- **Model:** `provider/model_name` (e.g., `gemini/gemini-2.5-flash`)
4. Save and start chatting
Expand All @@ -169,7 +170,7 @@ In your configuration file (e.g., `config.json`):
"title": "Gemini via Proxy",
"provider": "openai",
"model": "gemini/gemini-2.5-flash",
"apiBase": "http://127.0.0.1:8000/v1",
"apiBase": "http://127.0.0.1:7777/v1",
"apiKey": "your-proxy-api-key"
}
]
Expand All @@ -187,7 +188,7 @@ Claude Code natively supports custom Anthropic API endpoints. The recommended se
{
"env": {
"ANTHROPIC_AUTH_TOKEN": "your-proxy-api-key",
"ANTHROPIC_BASE_URL": "http://127.0.0.1:8000",
"ANTHROPIC_BASE_URL": "http://127.0.0.1:7777",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "gemini/gemini-3-pro",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "gemini/gemini-3-flash",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "openai/gpt-5-mini"
Expand All @@ -206,7 +207,7 @@ Now you can use Claude Code with Gemini, OpenAI, or any other configured provide
from anthropic import Anthropic

client = Anthropic(
base_url="http://127.0.0.1:8000",
base_url="http://127.0.0.1:7777",
api_key="your-proxy-api-key"
)

Expand Down Expand Up @@ -594,6 +595,62 @@ TIMEOUT_READ_NON_STREAMING=600 # Full response wait (10 min)

## OAuth Providers

<details>
<summary><b>Kiro CLI / Kiro IDE (Amazon Q / Kiro)</b></summary>

Supports multiple credential sources from Kiro IDE or kiro-cli.

**Setup (choose one):**

**Option 1: Direct Refresh Token** (best for Docker/stateless deployments)
```env
KIRO_REFRESH_TOKEN="your_kiro_refresh_token"
# or
REFRESH_TOKEN="your_kiro_refresh_token"

# Optional (only for Kiro Desktop auth, not needed for AWS SSO)
PROFILE_ARN="arn:aws:codewhisperer:us-east-1:..."
```

**Option 2: Kiro IDE (JSON credentials)** - Recommended for local use
1. Open Kiro IDE and sign in
2. The proxy auto-detects `~/.aws/sso/cache/kiro-auth-token.json`
```env
# Or specify explicitly:
KIRO_CREDS_FILE="~/.aws/sso/cache/kiro-auth-token.json"
```

**Option 3: kiro-cli (SQLite database)**
1. Install and log in with `kiro-cli`
2. The proxy auto-detects the SQLite database
```env
# Or specify explicitly:
KIRO_CLI_DB_FILE="~/Library/Application Support/kiro-cli/data.sqlite3"
```

**Common Options:**

```env
# Optional region override (default: us-east-1)
KIRO_REGION="us-east-1"
```

**Default auto-detection locations (checked in order):**

1. JSON: `~/.aws/sso/cache/kiro-auth-token.json`
2. SQLite (macOS): `~/Library/Application Support/kiro-cli/data.sqlite3`
3. SQLite (macOS): `~/Library/Application Support/amazon-q/data.sqlite3`
4. SQLite (Linux): `~/.local/share/kiro-cli/data.sqlite3`
5. SQLite (Linux): `~/.local/share/amazon-q/data.sqlite3`

**Authentication Types:**

The proxy automatically detects the auth type based on credentials:
- **Kiro Desktop Auth**: When `clientId`/`clientSecret` are NOT present (uses Kiro refresh endpoint)
- **AWS SSO OIDC**: When `clientId`/`clientSecret` ARE present (uses AWS OIDC endpoint)

</details>

<details>
<summary><b>Gemini CLI</b></summary>

Expand Down Expand Up @@ -799,7 +856,7 @@ python src/proxy_app/main.py [OPTIONS]

Options:
--host TEXT Host to bind (default: 0.0.0.0)
--port INTEGER Port to run on (default: 8000)
--port INTEGER Port to run on (default: 7777)
--enable-request-logging Enable detailed per-request logging
--add-credential Launch interactive credential setup tool
```
Expand Down Expand Up @@ -871,7 +928,7 @@ touch key_usage.json
docker run -d \
--name llm-api-proxy \
--restart unless-stopped \
-p 8000:8000 \
-p 7777:7777 \
-v $(pwd)/.env:/app/.env:ro \
-v $(pwd)/oauth_creds:/app/oauth_creds \
-v $(pwd)/logs:/app/logs \
Expand Down Expand Up @@ -945,7 +1002,7 @@ After=network.target
[Service]
Type=simple
WorkingDirectory=/path/to/LLM-API-Key-Proxy
ExecStart=/path/to/python -m uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 8000
ExecStart=/path/to/python -m uvicorn src.proxy_app.main:app --host 0.0.0.0 --port 7777
Restart=always

[Install]
Expand Down
10 changes: 9 additions & 1 deletion docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ services:
max-size: "10m"
max-file: "3"
ports:
- "${PORT:-8000}:8000"
- "${PORT:-7777}:7777"
volumes:
# Mount .env files for configuration
- ./.env:/app/.env:ro
Expand All @@ -23,6 +23,14 @@ services:
- ./key_usage.json:/app/key_usage.json
# Optionally mount additional .env files (e.g., combined credential files)
# - ./antigravity_all_combined.env:/app/antigravity_all_combined.env:ro

# --- Kiro CLI Credentials ---
# Option 1: Mount AWS SSO cache (JSON credentials from Kiro IDE) - RECOMMENDED
- ~/.aws/sso/cache:/root/.aws/sso/cache:ro
# Option 2: Mount kiro-cli SQLite database (macOS)
- ~/Library/Application Support/kiro-cli:/root/Library/Application Support/kiro-cli:ro
# Option 3: Mount kiro-cli SQLite database (Linux - uncomment if on Linux)
# - ~/.local/share/kiro-cli:/root/.local/share/kiro-cli:ro
environment:
# Skip OAuth interactive initialization in container (non-interactive)
- SKIP_OAUTH_INIT_CHECK=true
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.tls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ services:
max-size: "10m"
max-file: "3"
ports:
- "${PORT:-8000}:8000"
- "${PORT:-7777}:7777"
volumes:
# Mount .env files for configuration
- ./.env:/app/.env:ro
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ services:
max-size: "10m"
max-file: "3"
ports:
- "${PORT:-8000}:8000"
- "${PORT:-7777}:7777"
volumes:
# Mount .env files for configuration
- ./.env:/app/.env:ro
Expand Down
Loading
Loading