Skip to content

Commit cebcd93

Browse files
authored
Added a method to compute data drift on a ME (#170)
* Added a method to compute data drift on a ME * Added a basic explanation to DSSModelEvaluation.compute_data_drift() * Wording: input -> data (drift) * Allow both IDs and object instances to be used as reference for drift computation * Make full_model_like_id() public & available on both prediction & clustering models * Clarify that compute_data_drift() accepts either a Python object or an ID * full_model_like_id -> full_id * Auto-wait for future results in compute_data_drift() (by default)
1 parent ee8c2e7 commit cebcd93

File tree

2 files changed

+33
-0
lines changed

2 files changed

+33
-0
lines changed

dataikuapi/dss/ml.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1804,6 +1804,10 @@ def get_raw(self):
18041804
"""
18051805
return self.details
18061806

1807+
@property
1808+
def full_id(self):
1809+
return self.details["fullModelId"]
1810+
18071811
def get_raw_snippet(self):
18081812
"""
18091813
Gets the raw dictionary of trained model snippet.

dataikuapi/dss/modelevaluationstore.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33

44
from dataikuapi.dss.metrics import ComputedMetrics
55
from .discussion import DSSObjectDiscussions
6+
from .future import DSSFuture
67

78
from requests import utils
89

@@ -289,6 +290,34 @@ def delete(self):
289290
self.client._perform_json(
290291
"DELETE", "/projects/%s/modelevaluationstores/%s/runs/" % (self.project_key, self.mes_id), body=obj)
291292

293+
@property
294+
def full_id(self):
295+
return "ME-%s-%s-%s"%(self.project_key, self.mes_id, self.run_id)
296+
297+
def compute_data_drift(self, reference=None, data_drift_params=None, wait=True):
298+
"""
299+
Compute data drift against a reference model or model evaluation. The reference is determined automatically unless specified.
300+
301+
:param reference: saved model version (full ID or DSSTrainedPredictionModelDetails)
302+
or model evaluation (full ID or DSSModelEvaluation) to use as reference (optional)
303+
:type reference: Union[str, DSSModelEvaluation, DSSTrainedPredictionModelDetails]
304+
:param data_drift_params: data drift computation settings (optional)
305+
:param wait: data drift computation settings (optional)
306+
:returns: a `dict` containing data drift analysis results if `wait` is `True`, or a :class:`~dataikuapi.dss.future.DSSFuture` handle otherwise
307+
"""
308+
309+
if hasattr(reference, 'full_id'):
310+
reference = reference.full_id
311+
312+
future_response = self.client._perform_json(
313+
"POST", "/projects/%s/modelevaluationstores/%s/runs/%s/computeDataDrift" % (self.project_key, self.mes_id, self.run_id),
314+
body={
315+
"referenceId": reference,
316+
"dataDriftParams": data_drift_params
317+
})
318+
future = DSSFuture(self.client, future_response.get('jobId', None), future_response)
319+
return future.wait_for_result() if wait else future
320+
292321
def get_metrics(self):
293322
"""
294323
Get the metrics for this model evaluation. Metrics must be understood here as Metrics in DSS Metrics & Checks

0 commit comments

Comments
 (0)