Skip to content

Commit 35dc547

Browse files
lysnikolaouencukou
andauthored
gh-142518: Document thread-safety guarantees of dict operations (#144184)
* Address feedback; move thread safety section below see-also * Address feedback - don't mention equality comparison only * Change admonition to rubric; cross-reference glossary --------- Co-authored-by: Petr Viktorin <encukou@gmail.com>
1 parent 37430ca commit 35dc547

File tree

1 file changed

+140
-0
lines changed

1 file changed

+140
-0
lines changed

Doc/library/stdtypes.rst

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5593,6 +5593,146 @@ can be used interchangeably to index the same dictionary entry.
55935593
of a :class:`dict`.
55945594

55955595

5596+
.. _thread-safety-dict:
5597+
5598+
.. rubric:: Thread safety for dict objects
5599+
5600+
Creating a dictionary with the :class:`dict` constructor is atomic when the
5601+
argument to it is a :class:`dict` or a :class:`tuple`. When using the
5602+
:meth:`dict.fromkeys` method, dictionary creation is atomic when the
5603+
argument is a :class:`dict`, :class:`tuple`, :class:`set` or
5604+
:class:`frozenset`.
5605+
5606+
The following operations and functions are :term:`lock-free` and
5607+
:term:`atomic <atomic operation>`.
5608+
5609+
.. code-block::
5610+
:class: good
5611+
5612+
d[key] # dict.__getitem__
5613+
d.get(key) # dict.get
5614+
key in d # dict.__contains__
5615+
len(d) # dict.__len__
5616+
5617+
All other operations from here on hold the :term:`per-object lock`.
5618+
5619+
Writing or removing a single item is safe to call from multiple threads
5620+
and will not corrupt the dictionary:
5621+
5622+
.. code-block::
5623+
:class: good
5624+
5625+
d[key] = value # write
5626+
del d[key] # delete
5627+
d.pop(key) # remove and return
5628+
d.popitem() # remove and return last item
5629+
d.setdefault(key, v) # insert if missing
5630+
5631+
These operations may compare keys using :meth:`~object.__eq__`, which can
5632+
execute arbitrary Python code. During such comparisons, the dictionary may
5633+
be modified by another thread. For built-in types like :class:`str`,
5634+
:class:`int`, and :class:`float`, that implement :meth:`~object.__eq__` in C,
5635+
the underlying lock is not released during comparisons and this is not a
5636+
concern.
5637+
5638+
The following operations return new objects and hold the :term:`per-object lock`
5639+
for the duration of the operation:
5640+
5641+
.. code-block::
5642+
:class: good
5643+
5644+
d.copy() # returns a shallow copy of the dictionary
5645+
d | other # merges two dicts into a new dict
5646+
d.keys() # returns a new dict_keys view object
5647+
d.values() # returns a new dict_values view object
5648+
d.items() # returns a new dict_items view object
5649+
5650+
The :meth:`~dict.clear` method holds the lock for its duration. Other
5651+
threads cannot observe elements being removed.
5652+
5653+
The following operations lock both dictionaries. For :meth:`~dict.update`
5654+
and ``|=``, this applies only when the other operand is a :class:`dict`
5655+
that uses the standard dict iterator (but not subclasses that override
5656+
iteration). For equality comparison, this applies to :class:`dict` and
5657+
its subclasses:
5658+
5659+
.. code-block::
5660+
:class: good
5661+
5662+
d.update(other_dict) # both locked when other_dict is a dict
5663+
d |= other_dict # both locked when other_dict is a dict
5664+
d == other_dict # both locked for dict and subclasses
5665+
5666+
All comparison operations also compare values using :meth:`~object.__eq__`,
5667+
so for non-built-in types the lock may be released during comparison.
5668+
5669+
:meth:`~dict.fromkeys` locks both the new dictionary and the iterable
5670+
when the iterable is exactly a :class:`dict`, :class:`set`, or
5671+
:class:`frozenset` (not subclasses):
5672+
5673+
.. code-block::
5674+
:class: good
5675+
5676+
dict.fromkeys(a_dict) # locks both
5677+
dict.fromkeys(a_set) # locks both
5678+
dict.fromkeys(a_frozenset) # locks both
5679+
5680+
When updating from a non-dict iterable, only the target dictionary is
5681+
locked. The iterable may be concurrently modified by another thread:
5682+
5683+
.. code-block::
5684+
:class: maybe
5685+
5686+
d.update(iterable) # iterable is not a dict: only d locked
5687+
d |= iterable # iterable is not a dict: only d locked
5688+
dict.fromkeys(iterable) # iterable is not a dict/set/frozenset: only result locked
5689+
5690+
Operations that involve multiple accesses, as well as iteration, are never
5691+
atomic:
5692+
5693+
.. code-block::
5694+
:class: bad
5695+
5696+
# NOT atomic: read-modify-write
5697+
d[key] = d[key] + 1
5698+
5699+
# NOT atomic: check-then-act (TOCTOU)
5700+
if key in d:
5701+
del d[key]
5702+
5703+
# NOT thread-safe: iteration while modifying
5704+
for key, value in d.items():
5705+
process(key) # another thread may modify d
5706+
5707+
To avoid time-of-check to time-of-use (TOCTOU) issues, use atomic
5708+
operations or handle exceptions:
5709+
5710+
.. code-block::
5711+
:class: good
5712+
5713+
# Use pop() with default instead of check-then-delete
5714+
d.pop(key, None)
5715+
5716+
# Or handle the exception
5717+
try:
5718+
del d[key]
5719+
except KeyError:
5720+
pass
5721+
5722+
To safely iterate over a dictionary that may be modified by another
5723+
thread, iterate over a copy:
5724+
5725+
.. code-block::
5726+
:class: good
5727+
5728+
# Make a copy to iterate safely
5729+
for key, value in d.copy().items():
5730+
process(key)
5731+
5732+
Consider external synchronization when sharing :class:`dict` instances
5733+
across threads. See :ref:`freethreading-python-howto` for more information.
5734+
5735+
55965736
.. _dict-views:
55975737

55985738
Dictionary view objects

0 commit comments

Comments
 (0)