You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* POST `/v1/vector_stores/:vector_store_id/file_batches`
61
+
* GET `/v1/vector_stores/:vector_store_id/file_batches/:batch_id`
62
+
* POST `/v1/vector_stores/:vector_store_id/file_batches/:batch_id/cancel`
63
+
* GET `/v1/vector_stores/:vector_store_id/file_batches/:batch_id/files`
50
64
51
-
## task-queue
52
-
53
-
Task scheduling is needed in following sections:
54
-
55
-
* instinct-assistant
56
-
* Periodic task to check status of run objects and run step objects in background.
57
-
* FIFO queue for execution of run object, which handles agent execution. Only one running task is allowed for single run object.
58
-
59
-
### Worker queue for run objects
60
-
61
-
#### Preconditions
62
-
63
-
* run object is `queued` or `required_action`.
64
-
* all file resources needed are alive
65
-
66
-
#### Procedures
67
-
68
-
* Create `IAgentExecutor` instance with given tools setup
69
-
* Recover `AgentState` from database
70
-
* Run `IAgentExecutor::Stream` loop.
71
-
* If run object is `cancelling` or `expired`, then stop.
72
-
* Update status of run object to `in_progress`.
73
-
* if resolved step is agent thought, then
74
-
* create a message with thought text.
75
-
* create run step with status of `completed` and `step_details` with `message_creation` type.
76
-
* create another run step object with status of `in_progress`, create a message with `step_details` with `tool_call` type.
77
-
* if thought contains actions for function calls **stop**.
78
-
* if thought contains actions for other tool uses, then continue. As `code_interpreter` and `file_search` is triggered automatically.
79
-
* if resolved agent step is agent observation, then
80
-
* update `step_details` of last run step object.
81
-
* if resolved agent step is final message, then
82
-
* create a message object containing the final message text.
83
-
* create a run step with `step_details` of `message_creation` type and related `message_id`.
84
-
* and **stop**
85
-
* if stopped and
86
-
* status of run object is `cancelling`, then update status of run object to `canclled`.
87
-
* final message is generated, update status of run object to `completed`.
88
-
* function tool calls are required, then update status of run object to `requires_action`.
89
-
* error occurs during loop, update status of run object to `failed`.
90
-
91
-
92
-
#### Outcomes
93
-
* run object should be in intermediate status other than `queued`.
94
-
* run steps and generated messages are saved to database.
95
65
96
-
### Background queue for run objects
97
-
98
-
#### Preconditions
99
-
100
-
* Only one thread is running this task across entire cluster.
101
-
* Variables:
102
-
*`RUN_TIMEOUT` defaults to 10min.
103
-
104
-
105
-
#### Procedures
106
-
107
-
* Find all run objects that matches `modified_at < now() - RUN_TIMEOUT`.
108
-
* Loop run objects found
109
-
* Find last run step that are `in_progress` and update them to status of `expired`.
110
-
* Update run object to status of `expired`.
111
-
112
-
113
-
### Implementation notes
66
+
## task-queue
114
67
115
-
* An in-memory local queue is preferred in `mini-assistant`. [cameron314/concurrentqueue](https://github.com/cameron314/concurrentqueue) seems to be a good option.
116
-
* Uniform interface for task queue is needed for more scalable implementation on the cloud, where dedicated task scheduler will be used with multiple worker nodes setup. Possible options:
117
-
* Celery or other task-focused frameworks.
118
-
* Queue facilities in distributed compute frameworks like `ray`, e.g. [ray.util.queue.Queue](https://docs.ray.io/en/latest/ray-core/api/doc/ray.util.queue.Queue.html).
119
-
* Custom implementation based on message broker like Kafka.
68
+
In `mini-assistant`, an in-process, multi-consumer task queue is created. See [ThreadPoolTaskScheduler.hpp](../modules/instinct-data/include/task_scheduler/ThreadPoolTaskScheduler.hpp) for more details.
120
69
121
-
122
-
### Outcomes
70
+
Primary task handler classes are:
123
71
124
-
* task and task steps that are timeout have been marked as `expired`.
72
+
*[FileObjectTaskHandler.hpp](../modules/instinct-assistant/include/assistant/v2/task_handler/FileObjectTaskHandler.hpp): To process uploaded file.
73
+
*[RunObjectTaskHandler.hpp](../modules/instinct-assistant/include/assistant/v2/task_handler/RunObjectTaskHandler.hpp): To execute a run request for threads.
125
74
126
-
## `tool-server`
75
+
`ThreadPoolTaskScheduler` is kind of `ILifeCycle` and it's bootstrap in main.
127
76
128
-
### `file-search`
129
77
130
-
Considerations:
78
+
## tool-server
131
79
132
-
* duckdb implementation, mainly used for `mini-assistant`.
133
-
* For Each file object we will generate one document table and embedding table.
134
-
* Given only one `file_id` can be assigned to thread currently and only one process is accessing database files, we can manage all `VectorStorePtr` dynamically in memory. e.g a map from `file_id` to `VectorStorePtr`.
135
-
* a more scalable solution involves standalone vector database.
136
-
* A file object can be embedded and linked to a `collection`.
137
-
* Mapping from `file_id` and `collection`'s id is required.
80
+
### file-search
138
81
82
+
Primary classes:
139
83
140
-
Primary workflows:
84
+
*[SummaryGuidedFileSearch.hpp](../modules/instinct-assistant/include/assistant/v2/toolkit/SummaryGuidedFileSearch.hpp): Actual implementation of search tool
85
+
*[RunObjectTaskHandler.hpp](../modules/instinct-assistant/include/assistant/v2/task_handler/RunObjectTaskHandler.hpp): Bring the search tool to user's run requests.
141
86
142
-
1. File ingestion: operations in `FileBatch` and `File` endpoints will trigger `FileIngestionTaskHandler`, where file is split and transformed into embeddings.
143
-
2. Online search: `file-search` as built-in tools in run objects if explicitly requested.
0 commit comments