Bug: AgenticSession fail to start when created using CLI and an AgenticSession CR

[amoralej]

🐛 Bug Description

Ambient Code fail to create sessions using oc cli

Actual Behavior: What actually happened.

When I try to create an AgenticSession using cli with file:

apiVersion: vteam.ambient-code/v1alpha1
kind: AgenticSession
metadata:
  name: amber-health-test2
  namespace: watcher2
spec:
  initialPrompt: |
    Run nightly health check:
    1. Failed CI runs (past 24h)
    2. Security alerts
    3. Upstream breaking changes
    4. Open P0/P1 issues
    Store findings in session results (accessible via UI)
  repos:
      - url: https://github.com/amoralej/watcher-operator
        branch: main
        targetBranch: amber/health-test1
  autoPushOnComplete: true
  interactive: true

It fails with error:

4m36s (x3 over 4m40s)   Warning   Failed                  Pod/amber-health-test2-job-52bgl                             Error: secret "ambient-runner-token-amber-health-test2" not found

And actually the secret do not exist.

I find following messages in the operator log:

2025/12/19 09:00:54 Processing AgenticSession amber-health-test2 with phase Pending (desired: )
2025/12/19 09:00:54 Found ambient-vertex secret in ambient-code, copying to watcher2
2025/12/19 09:00:54 Successfully copied ambient-vertex secret to watcher2
2025/12/19 09:00:54 Langfuse disabled, skipping secret copy
2025/12/19 09:00:54 [Reconcile] Failed to add repo via runner: Post "http://session-amber-health-test2.watcher2.svc.cluster.local:8000/repos/add": dial tcp: lookup session-amber-health-test2.watcher2.svc.cluster.local on 10.217.4.10:53: no such host
2025/12/19 09:00:54 Vertex AI enabled, skipping runner secret ambient-runner-secrets validation
2025/12/19 09:00:54 Found ambient-non-vertex-integrations secret in watcher2, will inject as env vars
2025/12/19 09:00:54 Session amber-health-test2 initiated by user:  (userId: )
2025/12/19 09:00:54 Injecting integration secrets from 'ambient-non-vertex-integrations' for session amber-health-test2
2025/12/19 09:00:54 Skipping runner secrets 'ambient-runner-secrets' for session amber-health-test2 (Vertex enabled)
2025/12/19 09:00:54 Mounted ambient-vertex secret to /app/vertex in runner container for session amber-health-test2
2025/12/19 09:00:54 No Google OAuth secret found (session amber-health-test2), MCP Google Drive integration will not be available
2025/12/19 09:00:54 Created job amber-health-test2-job for AgenticSession amber-health-test2
2025/12/19 09:00:54 [DesiredPhase] Cleared desired-phase annotation after successful job creation
2025/12/19 09:00:54 Created AG-UI service session-amber-health-test2 for AgenticSession amber-health-test2
2025/12/19 09:00:54 Starting job monitoring for amber-health-test2-job (session: watcher2/amber-health-test2)
2025/12/19 09:00:54 Processing AgenticSession amber-health-test2 with phase Creating (desired: )
2025/12/19 09:00:54 Job amber-health-test2-job already being monitored, skipping duplicate
2025/12/19 09:00:54 Processing AgenticSession amber-health-test2 with phase Creating (desired: )
2025/12/19 09:00:54 Job amber-health-test2-job already being monitored, skipping duplicate
2025/12/19 09:00:59 Processing AgenticSession amber-health-test2 with phase Creating (desired: )
2025/12/19 09:00:59 Job amber-health-test2-job already being monitored, skipping duplicate
2025/12/19 09:01:04 Deleting copied ambient-vertex secret from namespace watcher2
2025/12/19 09:01:04 Stopped monitoring job amber-health-test2-job (goroutine exiting)
2025/12/19 09:01:04 Processing AgenticSession amber-health-test2 with phase Failed (desired: )

After that, if i go to the web UI and click on continue button, it starts properly and I see following messages in the operator log:

2025/12/19 09:10:37 Processing AgenticSession amber-health-test2 with phase Failed (desired: Running)
2025/12/19 09:10:37 [DesiredPhase] Session watcher2/amber-health-test2: user requested start/restart (current=Failed → desired=Running)
2025/12/19 09:10:37 [DesiredPhase] Continuation detected (parent=amber-health-test2), ensuring fresh runner token
2025/12/19 09:10:37 [TokenProvision] Regenerating runner token for watcher2/amber-health-test2
2025/12/19 09:10:37 [TokenProvision] Created secret ambient-runner-token-amber-health-test2 with runner token
2025/12/19 09:10:37 [TokenProvision] Successfully regenerated token for session watcher2/amber-health-test2
2025/12/19 09:10:37 [DesiredPhase] Session watcher2/amber-health-test2: set phase=Pending, will create job on next reconciliation
2025/12/19 09:10:37 Session continuation: reusing PVC ambient-workspace-amber-health-test2 from parent session amber-health-test2
2025/12/19 09:10:37 Found ambient-vertex secret in ambient-code, copying to watcher2
2025/12/19 09:10:37 Successfully copied ambient-vertex secret to watcher2
2025/12/19 09:10:37 Langfuse disabled, skipping secret copy
2025/12/19 09:10:37 Vertex AI enabled, skipping runner secret ambient-runner-secrets validation
2025/12/19 09:10:37 Found ambient-non-vertex-integrations secret in watcher2, will inject as env vars
2025/12/19 09:10:37 Session amber-health-test2 initiated by user:  (userId: )
2025/12/19 09:10:37 Session amber-health-test2: passing PARENT_SESSION_ID=amber-health-test2 to runner
2025/12/19 09:10:37 Injecting integration secrets from 'ambient-non-vertex-integrations' for session amber-health-test2
2025/12/19 09:10:37 Skipping runner secrets 'ambient-runner-secrets' for session amber-health-test2 (Vertex enabled)
2025/12/19 09:10:37 Mounted ambient-vertex secret to /app/vertex in runner container for session amber-health-test2
2025/12/19 09:10:37 No Google OAuth secret found (session amber-health-test2), MCP Google Drive integration will not be available
2025/12/19 09:10:37 Created job amber-health-test2-job for AgenticSession amber-health-test2
2025/12/19 09:10:37 [DesiredPhase] Cleared desired-phase annotation after successful job creation
2025/12/19 09:10:38 Created AG-UI service session-amber-health-test2 for AgenticSession amber-health-test2
2025/12/19 09:10:38 Starting job monitoring for amber-health-test2-job (session: watcher2/amber-health-test2)
2025/12/19 09:10:38 Processing AgenticSession amber-health-test2 with phase Creating (desired: )

And works properly.

Should create from CLI work as I tried?

Expected Behavior: What you expected to happen.

Sessions should be created using oc apply -f

Actual Behavior: What actually happened.

It fails as shown before

🌍 Environment

Version/Commit: [e.g. v1.2.3 or commit hash]

containers with latest tag from Dec 18th 25

For vteam_operator:

   {
          "Id": "eb96a0562a795e78570ea5b686b9e2a7e502af0785b19004006b30408947dc06",
          "Digest": "sha256:fd3279c868692daafdf9ca2849a1b44eb3f9ffa3617d9f1ee477dd84dd5a2a5a",
          "RepoTags": [
               "quay.io/ambient_code/vteam_operator:latest"
          ],
          "RepoDigests": [
               "quay.io/ambient_code/vteam_operator@sha256:0b12bd0d6c6e0b19e755af5e6080cb3742393a227328b0f1f7b8bc4c8d16d4d6",
               "quay.io/ambient_code/vteam_operator@sha256:fd3279c868692daafdf9ca2849a1b44eb3f9ffa3617d9f1ee477dd84dd5a2a5a"
          ],
          "Parent": "",
          "Comment": "",
          "Created": "2025-12-18T05:37:53.079466395Z",

Kubernetes Version: [if applicable - e.g. 1.28.2]

OpenShift 4.18

📋 Additional Context

Using vertex AI configuration

Deployed using make deploy on a CRC environment

🏷️ Labels

• Priority: [low/medium/high/critical]
• Complexity: [trivial/easy/medium/hard]
• Component: [frontend/backend/operator/tools/docs]