Skip to content

Commit 7c701af

Browse files
authored
chore: upgrade dreadnode sdk to 1.17.1 and add org/workspace params (#137)
- Upgrade dreadnode SDK from 1.14.1 to 1.17.1 to fix user bucket data issues with new platform - Add organization and workspace parameters to DreadnodeArgs - Pass organization and workspace to dn.configure() - Update README with organization and workspace documentation
1 parent ed1a4f8 commit 7c701af

File tree

4 files changed

+1757
-1768
lines changed

4 files changed

+1757
-1768
lines changed

README.md

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,45 @@ uv run -m airtbench --help
8787
### Basic Usage
8888

8989
```bash
90-
uv run -m airtbench --model $MODEL --project $PROJECT --platform-api-key $DREADNODE_TOKEN --token $DREADNODE_TOKEN --server https://platform.dreadnode.io --max-steps 100 --inference_timeout 240 --enable-cache --no-give-up --challenges bear1 bear2
90+
uv run -m airtbench \
91+
--model $MODEL \
92+
--project $PROJECT \
93+
--platform-api-key $DREADNODE_TOKEN \
94+
--token $DREADNODE_TOKEN \
95+
--server https://platform.dreadnode.io \
96+
--organization $ORGANIZATION \
97+
--max-steps 100 \
98+
--inference-timeout 240 \
99+
--enable-cache \
100+
--no-give-up \
101+
--challenges bear1 bear2
102+
```
103+
104+
**Organization and Workspace Parameters**
105+
106+
If you are part of multiple organizations, you must specify which organization to use:
107+
108+
```bash
109+
--organization "dreadnode"
110+
```
111+
112+
Optionally, you can also specify a workspace within the organization:
113+
114+
```bash
115+
--organization "dreadnode" --workspace "my-workspace"
116+
```
117+
118+
**Example with organization:**
119+
120+
```bash
121+
uv run -m airtbench \
122+
--model openai/gpt-4o \
123+
--project airtbench \
124+
--platform-api-key $DREADNODE_TOKEN \
125+
--token $DREADNODE_TOKEN \
126+
--server https://platform.dreadnode.io \
127+
--organization "dreadnode" \
128+
--challenges bear1
91129
```
92130

93131
### Challenge Filtering

airtbench/main.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,10 @@ class DreadnodeArgs:
8484
"""Local directory to store data in"""
8585
project: str = "airtbench"
8686
"""Project name"""
87+
organization: str | None = None
88+
"""Organization name"""
89+
workspace: str | None = None
90+
"""Workspace name"""
8791

8892

8993
# Models
@@ -709,6 +713,8 @@ async def main(
709713
token=dn_args.token,
710714
local_dir=dn_args.local_dir or False,
711715
project=dn_args.project,
716+
organization=dn_args.organization,
717+
workspace=dn_args.workspace,
712718
console=True,
713719
)
714720

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ dependencies = [
77
"backoff>=2.2.1",
88
"cyclopts>=3.12.0",
99
"docker>=7.1.0",
10-
"dreadnode==1.14.1",
10+
"dreadnode==1.17.1",
1111
"ipykernel>=6.29.5",
1212
"mypy>=1.15.0",
1313
"pythonnet>=3.0.5",

0 commit comments

Comments
 (0)