Skip to content

Commit 12dbae4

Browse files
authored
gh-142518: Define lock-free and per-object lock (#144548)
- Add definitions of lock-free and per-object lock to the glossary - Cross-reference these from list thread safety notes - Change admonition to rubric
1 parent 17ab556 commit 12dbae4

File tree

2 files changed

+96
-73
lines changed

2 files changed

+96
-73
lines changed

Doc/glossary.rst

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -951,6 +951,16 @@ Glossary
951951
to locks exist such as queues, producer/consumer patterns, and
952952
thread-local state. See also :term:`deadlock`, and :term:`reentrant`.
953953

954+
lock-free
955+
An operation that does not acquire any :term:`lock` and uses atomic CPU
956+
instructions to ensure correctness. Lock-free operations can execute
957+
concurrently without blocking each other and cannot be blocked by
958+
operations that hold locks. In :term:`free-threaded <free threading>`
959+
Python, built-in types like :class:`dict` and :class:`list` provide
960+
lock-free read operations, which means other threads may observe
961+
intermediate states during multi-step modifications even when those
962+
modifications hold the :term:`per-object lock`.
963+
954964
loader
955965
An object that loads a module.
956966
It must define the :meth:`!exec_module` and :meth:`!create_module` methods
@@ -1217,6 +1227,16 @@ Glossary
12171227
<faq-argument-vs-parameter>`, the :class:`inspect.Parameter` class, the
12181228
:ref:`function` section, and :pep:`362`.
12191229

1230+
per-object lock
1231+
A :term:`lock` associated with an individual object instance rather than
1232+
a global lock shared across all objects. In :term:`free-threaded
1233+
<free threading>` Python, built-in types like :class:`dict` and
1234+
:class:`list` use per-object locks to allow concurrent operations on
1235+
different objects while serializing operations on the same object.
1236+
Operations that hold the per-object lock prevent other locking operations
1237+
on the same object from proceeding, but do not block :term:`lock-free`
1238+
operations.
1239+
12201240
path entry
12211241
A single location on the :term:`import path` which the :term:`path
12221242
based finder` consults to find modules for importing.

Doc/library/stdtypes.rst

Lines changed: 76 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1441,108 +1441,111 @@ application).
14411441
list appear empty for the duration, and raises :exc:`ValueError` if it can
14421442
detect that the list has been mutated during a sort.
14431443

1444-
.. admonition:: Thread safety
1444+
.. _thread-safety-list:
14451445

1446-
Reading a single element from a :class:`list` is
1447-
:term:`atomic <atomic operation>`:
1446+
.. rubric:: Thread safety for list objects
14481447

1449-
.. code-block::
1450-
:class: green
1448+
Reading a single element from a :class:`list` is
1449+
:term:`atomic <atomic operation>`:
14511450

1452-
lst[i] # list.__getitem__
1451+
.. code-block::
1452+
:class: green
14531453
1454-
The following methods traverse the list and use :term:`atomic <atomic operation>`
1455-
reads of each item to perform their function. That means that they may
1456-
return results affected by concurrent modifications:
1454+
lst[i] # list.__getitem__
14571455
1458-
.. code-block::
1459-
:class: maybe
1456+
The following methods traverse the list and use :term:`atomic <atomic operation>`
1457+
reads of each item to perform their function. That means that they may
1458+
return results affected by concurrent modifications:
14601459

1461-
item in lst
1462-
lst.index(item)
1463-
lst.count(item)
1460+
.. code-block::
1461+
:class: maybe
14641462
1465-
All of the above methods/operations are also lock-free. They do not block
1466-
concurrent modifications. Other operations that hold a lock will not block
1467-
these from observing intermediate states.
1463+
item in lst
1464+
lst.index(item)
1465+
lst.count(item)
14681466
1469-
All other operations from here on block using the per-object lock.
1467+
All of the above operations avoid acquiring :term:`per-object locks
1468+
<per-object lock>`. They do not block concurrent modifications. Other
1469+
operations that hold a lock will not block these from observing intermediate
1470+
states.
14701471

1471-
Writing a single item via ``lst[i] = x`` is safe to call from multiple
1472-
threads and will not corrupt the list.
1472+
All other operations from here on block using the :term:`per-object lock`.
14731473

1474-
The following operations return new objects and appear
1475-
:term:`atomic <atomic operation>` to other threads:
1474+
Writing a single item via ``lst[i] = x`` is safe to call from multiple
1475+
threads and will not corrupt the list.
14761476

1477-
.. code-block::
1478-
:class: good
1477+
The following operations return new objects and appear
1478+
:term:`atomic <atomic operation>` to other threads:
14791479

1480-
lst1 + lst2 # concatenates two lists into a new list
1481-
x * lst # repeats lst x times into a new list
1482-
lst.copy() # returns a shallow copy of the list
1480+
.. code-block::
1481+
:class: good
14831482
1484-
Methods that only operate on a single elements with no shifting required are
1485-
:term:`atomic <atomic operation>`:
1483+
lst1 + lst2 # concatenates two lists into a new list
1484+
x * lst # repeats lst x times into a new list
1485+
lst.copy() # returns a shallow copy of the list
14861486
1487-
.. code-block::
1488-
:class: good
1487+
The following methods that only operate on a single element with no shifting
1488+
required are :term:`atomic <atomic operation>`:
14891489

1490-
lst.append(x) # append to the end of the list, no shifting required
1491-
lst.pop() # pop element from the end of the list, no shifting required
1490+
.. code-block::
1491+
:class: good
14921492
1493-
The :meth:`~list.clear` method is also :term:`atomic <atomic operation>`.
1494-
Other threads cannot observe elements being removed.
1493+
lst.append(x) # append to the end of the list, no shifting required
1494+
lst.pop() # pop element from the end of the list, no shifting required
14951495
1496-
The :meth:`~list.sort` method is not :term:`atomic <atomic operation>`.
1497-
Other threads cannot observe intermediate states during sorting, but the
1498-
list appears empty for the duration of the sort.
1496+
The :meth:`~list.clear` method is also :term:`atomic <atomic operation>`.
1497+
Other threads cannot observe elements being removed.
14991498

1500-
The following operations may allow lock-free operations to observe
1501-
intermediate states since they modify multiple elements in place:
1499+
The :meth:`~list.sort` method is not :term:`atomic <atomic operation>`.
1500+
Other threads cannot observe intermediate states during sorting, but the
1501+
list appears empty for the duration of the sort.
15021502

1503-
.. code-block::
1504-
:class: maybe
1503+
The following operations may allow :term:`lock-free` operations to observe
1504+
intermediate states since they modify multiple elements in place:
15051505

1506-
lst.insert(idx, item) # shifts elements
1507-
lst.pop(idx) # idx not at the end of the list, shifts elements
1508-
lst *= x # copies elements in place
1506+
.. code-block::
1507+
:class: maybe
15091508
1510-
The :meth:`~list.remove` method may allow concurrent modifications since
1511-
element comparison may execute arbitrary Python code (via
1512-
:meth:`~object.__eq__`).
1509+
lst.insert(idx, item) # shifts elements
1510+
lst.pop(idx) # idx not at the end of the list, shifts elements
1511+
lst *= x # copies elements in place
15131512
1514-
:meth:`~list.extend` is safe to call from multiple threads. However, its
1515-
guarantees depend on the iterable passed to it. If it is a :class:`list`, a
1516-
:class:`tuple`, a :class:`set`, a :class:`frozenset`, a :class:`dict` or a
1517-
:ref:`dictionary view object <dict-views>` (but not their subclasses), the
1518-
``extend`` operation is safe from concurrent modifications to the iterable.
1519-
Otherwise, an iterator is created which can be concurrently modified by
1520-
another thread. The same applies to inplace concatenation of a list with
1521-
other iterables when using ``lst += iterable``.
1513+
The :meth:`~list.remove` method may allow concurrent modifications since
1514+
element comparison may execute arbitrary Python code (via
1515+
:meth:`~object.__eq__`).
15221516

1523-
Similarly, assigning to a list slice with ``lst[i:j] = iterable`` is safe
1524-
to call from multiple threads, but ``iterable`` is only locked when it is
1525-
also a :class:`list` (but not its subclasses).
1517+
:meth:`~list.extend` is safe to call from multiple threads. However, its
1518+
guarantees depend on the iterable passed to it. If it is a :class:`list`, a
1519+
:class:`tuple`, a :class:`set`, a :class:`frozenset`, a :class:`dict` or a
1520+
:ref:`dictionary view object <dict-views>` (but not their subclasses), the
1521+
``extend`` operation is safe from concurrent modifications to the iterable.
1522+
Otherwise, an iterator is created which can be concurrently modified by
1523+
another thread. The same applies to inplace concatenation of a list with
1524+
other iterables when using ``lst += iterable``.
15261525

1527-
Operations that involve multiple accesses, as well as iteration, are never
1528-
atomic. For example:
1526+
Similarly, assigning to a list slice with ``lst[i:j] = iterable`` is safe
1527+
to call from multiple threads, but ``iterable`` is only locked when it is
1528+
also a :class:`list` (but not its subclasses).
15291529

1530-
.. code-block::
1531-
:class: bad
1530+
Operations that involve multiple accesses, as well as iteration, are never
1531+
atomic. For example:
15321532

1533-
# NOT atomic: read-modify-write
1534-
lst[i] = lst[i] + 1
1533+
.. code-block::
1534+
:class: bad
15351535
1536-
# NOT atomic: check-then-act
1537-
if lst:
1538-
item = lst.pop()
1536+
# NOT atomic: read-modify-write
1537+
lst[i] = lst[i] + 1
15391538
1540-
# NOT thread-safe: iteration while modifying
1541-
for item in lst:
1542-
process(item) # another thread may modify lst
1539+
# NOT atomic: check-then-act
1540+
if lst:
1541+
item = lst.pop()
15431542
1544-
Consider external synchronization when sharing :class:`list` instances
1545-
across threads. See :ref:`freethreading-python-howto` for more information.
1543+
# NOT thread-safe: iteration while modifying
1544+
for item in lst:
1545+
process(item) # another thread may modify lst
1546+
1547+
Consider external synchronization when sharing :class:`list` instances
1548+
across threads. See :ref:`freethreading-python-howto` for more information.
15461549

15471550

15481551
.. _typesseq-tuple:

0 commit comments

Comments
 (0)