You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/models/models.md
+72-20Lines changed: 72 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,4 @@
1
-
Ths is a possible implementation of AI model support in AppLab.
1
+
This is a possible implementation of AI model support in AppLab.
2
2
3
3
# Models
4
4
In AI, a model is a trained system able to recognize patterns and/or make predictions.
@@ -22,14 +22,15 @@ models/
22
22
build#2
23
23
```
24
24
25
-
*NOTE:* We need to discuss how to handle different builds of the same model. In this case, we must ensure we reference the specific build to be run and manage the models' and model definitions' state and deletion accordingly.
25
+
*NOTE:* We need to eventually discuss how to handle different builds of the same model. In this case, we must ensure we reference the specific build to be run and manage the model state, model definition state, and deletion accordingly.
26
26
27
27
*NOTE:* To be discussed how to map Edge Impulse in an AppLab model. See Edge Impulse section below.
28
28
29
-
The ```conf.yaml``` file contains generic model information. The application expects this data to be available for all models.
29
+
The ```conf.yaml``` file contains generic model information. The application expects this data to be available for all models. The ```model_categories``` is a list of the categories of the model.
30
30
```
31
31
runner: brick
32
32
name : "Glass breaking classifier"
33
+
model_categories: [audio, video]
33
34
model_labels:
34
35
- "Background"
35
36
- "Glass_Breaking"
@@ -50,8 +51,8 @@ In AI, a category is a common way to classify AI models, indicating the type of
50
51
51
52
In AppLab, a category is defined as a keyword used to connect models to bricks. It defines which models can be attached to which bricks. Both a model and a brick have a list of categories.
52
53
53
-
### Use Case (Linking Models to Bricks)
54
-
Use Case (link the Figma here): In the brick configuration, we can choose the model to be used with the brick. The provided list is a filtered list of all models where the intersection of the model's category list and the brick's category list is not empty.
54
+
### Linking Models to Bricks
55
+
In the brick configuration, we can choose the model to be used with the brick. The provided list is a filtered list of all models where the intersection of the model's category list and the brick's category list is not empty.
55
56
56
57
**We assume here that every model with a given category is able to work with any provider.** If there is a special model that is only able to work with a specific brick, we can always add a custom category to associate the model with the working brick.
57
58
@@ -62,12 +63,11 @@ A brick is defined in the brick-list.yaml
62
63
```
63
64
- id: arduino:audio_classification
64
65
name: Audio Classification
65
-
category: audio, other_cat
66
-
default_model_name: arduino:glass-breaking
66
+
categories: audio, other_cat
67
67
```
68
-
The ```default_model_name``` field was added to specify a default model.
69
68
70
-
*NOTE:* What should be done if this is a custom model? The concept of a default custom model does not exist.
69
+
*NOTE:* categories is a model specific variable, what to do for a brick that does not involve a model?
70
+
Somewhere is null, somewhere is "storage"
71
71
72
72
## App.yaml
73
73
This file contains the app's information and configuration.
*Note:* The following variables are defined in the current implementation but should be removed because they must be hidden from the final user.
224
-
We need to discuss whether these variables should be defined here, or if we should leave them hardcoded instead (is this the correct alternative approach?).
## Case #5 Two distinct bricks using the same custom model
375
+
This case is the same as Case #3, where the variables used to customize the model are configurable by the user, with an optional default value.
375
376
376
377
The following section explores whether this model is suitable for specific use cases.
377
378
378
-
## The user chooses a model for a brick (link Figma here)
379
+
## Case #6The user chooses a model for a brick (link Figma here)
379
380
In the brick configuration, there is a list of compatible models and the user selects the ```dog-detector``` model.
380
381
In this case, the model will be stored in ```App.yaml```. The brick section will look like:
381
382
```
@@ -385,7 +386,7 @@ bricks:
385
386
model: dog-detector
386
387
- arduino:mood_detector: {}
387
388
```
388
-
## We need to know if a model build or a model definition can be deleted (link the Figma here):
389
+
## Case #7We need to know if a model build or a model definition can be deleted (link the Figma here):
389
390
To delete a build oir a model we need to know the state of the model in the system:
390
391
***Downloaded** (present on the board and available for a brick to be used)
391
392
***Installed** At least one brick is referencing the model.
@@ -399,7 +400,7 @@ The Installed state can be inferred by checking all application ```App.yaml``` f
399
400
We can provide a list of models, their states, and all the builds on the File System (FS) (if we implement different versions/builds).
400
401
We can allow the deletion of only specific builds or the entire model structure.
401
402
402
-
## We want to support a different model/executor
403
+
## Case #8We want to support a different model/executor
403
404
We aim to support a learning model, where the model is defined by a set of weights and generated by a framework like TensorFlow. We intend to deploy and run this model within a web browser using JavaScript (TensorFlow.js).
404
405
405
406
In this scenario, we can declare the model using its folder structure (API requests can be developed if required) and the necessary YAML files.
@@ -414,7 +415,7 @@ The question here is how to make the model variables available to the container.
414
415
415
416
2. Environment Variables: Alternatively, we can make the entire model configuration available to the container as environment variables.
416
417
417
-
# Questions:
418
+
# Questions to be discussed:
418
419
1. Should we keep "audio" as a generic category instead of using something more specific, such as ```ei-audio``` or ```qualcomm-audio```?
419
420
At this level, this has no relevance.
420
421
@@ -428,12 +429,63 @@ If this becomes technically impossible in the future, we can then add a more spe
428
429
3. Why there is the
429
430
```require model``` in the brick definition?
430
431
431
-
4.Check the use case of the Figma where we go to EI and come back with a model
432
+
4.Variable Configuration: Should variables related to a model be placed in the model or in the brick?
432
433
433
-
# Proposed Changes to Bricks and Models Configuration
434
-
This section details the proposed changes to the current implementation.
434
+
Since different bricks may use different variables to enable the container engine to start the AI engine, it is better to do not declare these variables in the model definition.
435
435
436
+
When the user provides custom values, the variable definitions will be placed in the App.yaml file. See Use Case #3 App.yaml below, where one brick requires:
5. Check the use case of the Figma where we go to EI and come back with a model
460
+
461
+
# Proposed Changes to Bricks and Models Configuration
436
462
A Model definition contains a list of categories.
437
463
438
-
A Brick definition containes a list of supported categories supported and a default_mode_name.
439
-
464
+
A Brick definition containes a list of supported categories and a default_model_name.
465
+
466
+
# Variables
467
+
This is a cross-cutting issue, not strictly related to models, but it's worth spending a moment here because it affects how models and bricks are managed.
468
+
469
+
In the current implementation, all variables originating from the following sources are collected into a single map:
470
+
- Variables declared in the brick with a default model.
471
+
- Variables defined in the model.
472
+
- Variables defined or overridden by the user in the App.yaml file.
473
+
This map is then passed directly to the ```environment:``` section of the Compose file.
474
+
475
+
If a variable can be overridden by the user, it must be declared in the brick definition, it is not required or mandatory to declare
476
+
the other variables that can not be visible/overrided by the user.
477
+
478
+
*NOTE:* Due to a recent issue where using an empty string as a default value is possible, the proposal is to add the is_required flag to the brick definition file, as follows:
479
+
```
480
+
- name: MY_VARIABLE
481
+
description: Variable with empty string value as default
482
+
default: ""
483
+
required: true
484
+
```
485
+
486
+
*NOTE:* Another proposal is to add all required brick variables to the brick definition file. The drawback is that the brick file could potentially become large. The advantage is that all variables are visible, which reduces the risk of conflicts when adding new variables to existing projects.
0 commit comments