Skip to content

Commit 2f3ac4d

Browse files
committed
feat: skip Stripe billing in evals by setting CI=true
Instead of adding a new SKIP_BILLING env var, evals now set CI=true at the top of their entry points (main.ts, main-nightly.ts, main-single-eval.ts, main-hard-tasks.ts). This ensures billing is always skipped when running evals, without requiring users to remember to set an env var. The shouldAttemptStripeMetering() function already checks CI=true, so this approach is simpler and follows the existing pattern.
1 parent ca8a8f1 commit 2f3ac4d

File tree

5 files changed

+14
-1
lines changed

5 files changed

+14
-1
lines changed

evals/buffbench/main-hard-tasks.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
// Set CI=true to skip Stripe billing during evals
2+
process.env.CI = 'true'
3+
14
import fs from 'fs'
25
import path from 'path'
36

evals/buffbench/main-nightly.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
// Set CI=true to skip Stripe billing during evals
2+
process.env.CI = 'true'
3+
14
import path from 'path'
25

36
import { sendBasicEmail } from '@codebuff/internal/loops'

evals/buffbench/main-single-eval.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
// Set CI=true to skip Stripe billing during evals
2+
process.env.CI = 'true'
3+
14
import path from 'path'
25

36
import { runBuffBench } from './run-buffbench'

evals/buffbench/main.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
// Set CI=true to skip Stripe billing during evals
2+
process.env.CI = 'true'
3+
14
import path from 'path'
25

36
import { runBuffBench } from './run-buffbench'

packages/billing/src/stripe-metering.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ const STRIPE_METER_EVENT_NAME = 'credits'
1010
const STRIPE_METER_REQUEST_TIMEOUT_MS = 10_000
1111

1212
function shouldAttemptStripeMetering(): boolean {
13-
// Avoid sending Stripe metering events in CI and when Stripe isn't configured.
13+
// Avoid sending Stripe metering events in CI or when Stripe isn't configured.
14+
// Evals set CI=true to skip billing.
1415
if (process.env.CI === 'true' || process.env.CI === '1') return false
1516
return Boolean(process.env.STRIPE_SECRET_KEY)
1617
}

0 commit comments

Comments
 (0)