You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+38-4Lines changed: 38 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,43 @@
7
7
***Gains and Lift Charts**
8
8
***Decision Curves**
9
9
10
-
The library is designed to be easy to use, while still offering a high degree of control over the final plots.
10
+
The library is designed to be easy to use, while still offering a high degree of control over the final plots. For some reproducible examples please visit the [rtichoke blog](https://uriahf.github.io/rtichoke-py/blog.html)!
11
+
12
+
## Installation
13
+
14
+
You can install `rtichoke` from PyPI:
15
+
16
+
```bash
17
+
pip install rtichoke
18
+
```
19
+
20
+
## Getting Started
21
+
22
+
To use `rtichoke`, you'll need two main inputs:
23
+
24
+
*`probs`: A dictionary containing your model's predicted probabilities.
25
+
*`reals`: A dictionary of the true binary outcomes.
26
+
27
+
Here's a quick example of how to create a ROC curve for a single model:
28
+
29
+
```python
30
+
import numpy as np
31
+
import rtichoke as rk
32
+
33
+
# Sample data for a model. Note that the probabilities for the
34
+
# positive class (1) are generally higher than for the negative class (0).
@@ -18,6 +54,4 @@ The library is designed to be easy to use, while still offering a high degree of
18
54
19
55
## Documentation
20
56
21
-
For a complete guide to the library, including a "Getting Started" tutorial and a full API reference, please see the **[official documentation](https://your-documentation-url.com)**.
22
-
23
-
*(Note: The documentation URL will need to be updated once the website is deployed.)*
57
+
For a complete guide to the library, including a "Getting Started" tutorial and a full API reference, please see the **[official documentation](https://uriahf.github.io/rtichoke-py/)**.
This tutorial provides a basic introduction to the `rtichoke` library. We'll walk through the process of preparing data, creating a decision curve, and visualizing the results.
5
+
This tutorial provides an introduction to the `rtichoke` library, showing how to visualize model performance for different scenarios.
6
6
7
7
## 1. Import Libraries
8
8
@@ -13,50 +13,89 @@ import numpy as np
13
13
import rtichoke as rk
14
14
```
15
15
16
-
## 2. Prepare Your Data
16
+
## 2. Understanding the Inputs
17
17
18
-
`rtichoke` expects data in a specific format. You'll need two main components:
18
+
`rtichoke` expects two main inputs for creating performance curves:
19
19
20
-
***Probabilities (`probs`)**: A dictionary where keys are model names and values are NumPy arrays of predicted probabilities.
21
-
***Real Outcomes (`reals`)**: A NumPy array containing the true binary outcomes (0 or 1).
20
+
***`probs` (Probabilities)**: A dictionary where keys are model or population names and values are lists or NumPy arrays of predicted probabilities.
21
+
***`reals` (Outcomes)**: A dictionary where keys are population names and values are lists or NumPy arrays of the true binary outcomes (0 or 1).
22
22
23
-
Let's create some sample data for two different models:
23
+
Let's look at the three main use cases.
24
+
25
+
### Use Case 1: Single Model
26
+
27
+
This is the simplest case, where you want to evaluate the performance of a single predictive model.
28
+
29
+
For this, you provide `probs` with a single entry for your model and `reals` with a single entry for the corresponding outcomes.
# In an interactive environment (like a Jupyter notebook),
44
+
# this will display the plot.
45
+
fig.show()
40
46
```
41
47
42
-
## 3. Create a Decision Curve
48
+
### Use Case 2: Models Comparison
49
+
50
+
Often, you want to compare the performance of several different models on the *same* population.
43
51
44
-
Now that we have our data, we can create a decision curve. This is a simple one-liner with `rtichoke`:
52
+
For this, you provide `probs` with an entry for each model you want to compare. `reals` will still have a single entry, since the outcome data is the same for all models.
45
53
46
54
```python
47
-
fig = rk.create_decision_curve(
48
-
probs=probs_dict,
49
-
reals=reals,
55
+
# Sample data for two models. Model A is better at separating the classes.
# Create a precision-recall curve to compare the models
65
+
fig = rk.create_precision_recall_curve(
66
+
probs=probs_comparison,
67
+
reals=reals_comparison,
50
68
)
69
+
70
+
fig.show()
51
71
```
52
72
53
-
##4. Show the Plot
73
+
### Use Case 3: Several Populations
54
74
55
-
Finally, let's display the plot. Since `rtichoke` uses Plotly under the hood, you can show the figure just like any other Plotly object.
75
+
This is useful when you want to evaluate a single model's performance across different populations. A common example is comparing performance on a training set versus a testing set to check for overfitting.
76
+
77
+
For this, you provide `probs` with an entry for each population and `reals` with a corresponding entry for each population's outcomes.
56
78
57
79
```python
58
-
# To display the plot in an interactive environment (like a Jupyter notebook)
80
+
# Sample data for a train and test set.
81
+
# The model performs slightly better on the train set.
"Test": np.array([0, 1, 0, 1, 0, 0]) # Note one outcome is different
89
+
}
90
+
91
+
# Create a calibration curve to compare the model's performance
92
+
# on the two populations.
93
+
fig = rk.create_calibration_curve(
94
+
probs=probs_populations,
95
+
reals=reals_populations,
96
+
)
97
+
59
98
fig.show()
60
99
```
61
100
62
-
And that's it! You've created your first decision curve with `rtichoke`. From here, you can explore the other curve types and options that the library has to offer.
101
+
And that's it! You've now seen how to create three of the most common evaluation plots with `rtichoke`. From here, you can explore the other curve types and options that the library has to offer in the [API Reference](../reference/index.qmd).
0 commit comments