Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,19 @@ ChristianModel was trained on the USPS_0-6 dataset. The model was trained for a
| Validation | 0.071 | 0.074 | 0.973 | 0.975 | 0.140 | 0.974 |
| Test | 0.247 | 0.096 | 0.931 | 0.934 | 0.134 | 0.932 |

### JohanModel & MNIST_4-9
This section reports the results from using the model "JohanModel" and the dataset MNIST_4-9 which contains MNIST digits from 4 to 9 (six classes in total).
All five available metrics were calculated for this experiment, model was trained for 12 epochs with learning rate of 0.001 and batch size 64.

The performance of the model is somewhat limited, at least compared with the results of JanModel on the other portion of MNIST dataset. Resulting metrics computed with macro-averaging below:

| Dataset Split | Loss | Entropy | Accuracy | Precision | Recall | F1 |
|---------------|-------|---------|----------|-----------|--------|-------|
| Train | 0.617 | 0.618 | 0.825 | 0.889 | 0.825 | 0.769 |
| Validation | 0.677 | 0.619 | 0.809 | 0.872 | 0.809 | 0.755 |
| Test | 0.679 | 0.618 | 0.810 | 0.870 | 0.810 | 0.755 |


## Citing
Please consider citing this repository if you end up using it for your work.
Several citation methods can be found under the "About" section.
Expand Down
20 changes: 19 additions & 1 deletion doc/Jan_page.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,22 @@ The class has the following methods:
* `__returnmetric__`: Returns the computed accuracy based on the averaging method for all stored predictions.
* `__reset__`: Resets the stored true and predicted labels.

The `forward` method takes the true labels and predicted labels as input and stores them. The `_macro_acc` method computes the macro-average accuracy by averaging the accuracy for each class. The `_micro_acc` method computes the micro-average accuracy by calculating the overall accuracy. The `__returnmetric__` method returns the computed accuracy based on the averaging method. The `__reset__` method resets the stored true and predicted labels to prepare for the next epoch.
The `forward` method takes the true labels and predicted labels as input and stores them. The `_macro_acc` method computes the macro-average accuracy by averaging the accuracy for each class. The `_micro_acc` method computes the micro-average accuracy by calculating the overall accuracy. The `__returnmetric__` method returns the computed accuracy based on the averaging method. The `__reset__` method resets the stored true and predicted labels to prepare for the next epoch.

## Challenges
### Running someone elses code

At the very beginning of the project we agreed upon a project structure that included wrappers for models, datasets and metrics which made it very easy to run other peoples code. Problems only arised in the development stage, when it was necessary to collect adjusments from all five participants in case of a change in the wrapper structure.

### Someone else running my code

Thanks to the project structure mentioned above it was also not difficult to have someone else handle the code I produced. It was sometimes necessary to make design adjustments on the go (for example having the metric wrapper pass the raw logits instead of predictions to the metric classes) and it was not always easy for me to catch up with all the important changes and adjust my code right away - sometimes an opened issue got closed before all the dependent code was adjusted. These problems were more and more sparse throughout the process as the whole group got more used to the workflow in a shared github repository.

### Tools

The course presented a lot of tools that were previously unknown to me. The whole concept of testing functions was never appreciated by me, until I saw its importance when working on a shared project, especially when combined with github actions automatic workflows. The sphinx documentation generation tool also prooved to be extremely useful, if only to keep track of all implemented classes.

I have migrated with all my work to the uv package manager after the course - it keeps the project environment local and it is much easier to use.

This was also my first serious experience with work on a shared project using github, so far I only used it as a personal repository or to share specific code.

1 change: 1 addition & 0 deletions doc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ This documentation lays out the code documentation of use for the FYS-8805 Colla
about.md
Magnus_page.md
Jan_page.md
Johan_page.md
:::