diff --git a/doc/Magnus_page.md b/doc/Magnus_page.md index c7c1d32..dd8a224 100644 --- a/doc/Magnus_page.md +++ b/doc/Magnus_page.md @@ -26,24 +26,19 @@ Each input is flattened over the channel, height and width channels. Then they a ## Entropy Metric In-Depth - -The EntropyPrediction class' main job is to take some inputs and return the Shannon Entropy metric of those inputs. The class has four methods with the following jobs: +The EntropyPrediction class' main job is to take some inputs from the MetricWrapper class and store the batchwise Shannon Entropy metric of those inputs. The class has four methods with the following jobs: * __init__ : Initialize the class. * __call__ : Main method which is used to calculate and store the batchwise shannon entropy. * __returnmetric__ : Returns the collected metric. * __reset__ : Removes all the stored values up until that point. Readies the instance for storing values from a new epoch. -The class is initialized with a single parameter called "averages". This is inspired from other PyTorch and NumPy implementations and controlls how values from different batches or within batches will be combined. The __init__ method checks the value of this argument with an assertion, which must be one of three string. We only allow "mean", "sum" and "none" as methods of combining the different entropy values. We'll come back to the specifics here. -Furthermore, this method will also store the different Shannon Entropy values as we pass values into the __call__ method. +The __init__ method has two arguments, both present for compatability issues. However, the num_classes argument is used as a check in the __call__ method to assert the input is of correctly assumed size. + +In __call__ we get both true labels and model logit scores for each sample in the batch as input. We're calculating Shannon Entropy, not KL-divergence, so the true labels aren't actually needed. +With permission I've used the scipy implementation to calculate entropy here. We apply a softmax over the logit values, then calculate the Shannon Entropy, and make sure to remove any Inf values which might arise from a perfect guess/distribution. + -In __call__ we get both true labels and model logit scores for each sample in the batch as input. We're calculating Shannon Entropy, not KL-divergence, so the true labels aren't needed. -With permission I've used the scipy implementation to calculate entropy here. We apply a softmax over the logit values, then calculate the Shannon Entropy, and make sure to remove any NaN or Inf values which might arise from a perfect guess/distribution. -Next we have the __returnmetric__ method which is used to retrive the stored metric. Here the averages argument comes into play. -Depending on what has been chosen as the averaging metric when initializing the class, one of the following operations will be applied to the stored values: -* Mean: Calculate the mean of the stored entropy values. -* Sum: Sum the stored entropy values. -* None: Do nothing with the stored entropy values. -Then the value(s) are returned. +Next we have the __returnmetric__ method which is used to retrive the stored metric. This returns the mean over all stored values. Effectively, this will return the average Shannon Entropy of the dataset. Lastly we have the __reset__ method which simply emptied the variable which stores the entropy values to prepare it for the next epoch. \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 1cb1da6..15b0167 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,7 +3,7 @@ name = "collaborative-coding-exam" version = "0.1.0" description = "Exam project in the collaborative coding course." readme = "README.md" -requires-python = ">=3.12" +requires-python = ">=3.11.5" dependencies = [ "black>=25.1.0", "h5py>=3.12.1",