Filter the records when given a minutes argument (#606)

* Fixes OOM issue when cleaning over 8M records

* Adding name to Authors

* Added changes to Changelog

* Removing redundant .all()

* Removing unnecessary typo.

* Filter records if given a minutes argument.

* Updated changelog

* Adding black formatting to unrelated files

* Removing commit, cleaning up changelog

Author: craigmaloney

CHANGES.rst

@@ -1,6 +1,10 @@
 Changes
 =======
 
+Unreleased
+----------
+- Add simple filtering if provided a minutes argument in `clean_duplicate_history` (gh-606)
+
 2.8.0 (2019-12-02)
 ------------------
 - Fixed `bulk_create_with_history support` for HistoryRecords with `relation_name` attribute (gh-591)

simple_history/management/commands/clean_duplicate_history.py

@@ -65,11 +65,17 @@ def _process(self, to_process, date_back=None, dry_run=True):
             if not found:
                 continue
 
-            # it would be great if we could just iterate over the instances that
-            # have changes (in the given period) but
-            # `m_qs.values(model._meta.pk.name).distinct()`
-            # is actually slower than looping all and filtering in the code...
-            for o in model.objects.iterator():
+            # Break apart the query so we can add additional filtering
+            model_query = model.objects.all()
+
+            # If we're provided a stop date take the initial hit of getting the
+            # filtered records to iterate over
+            if stop_date:
+                model_query = model_query.filter(
+                    pk__in=(m_qs.values_list(model._meta.pk.name).distinct())
+                )
+
+            for o in model_query.iterator():
                 self._process_instance(o, model, stop_date=stop_date, dry_run=dry_run)
 
     def _process_instance(self, instance, model, stop_date=None, dry_run=True):