Merge branch 'main' into add_test_xpickle

Author: serhiy-storchaka

Doc/library/compression.zstd.rst

@@ -73,7 +73,7 @@ Reading and writing compressed files
    argument is not None, a :exc:`!TypeError` will be raised.
 
    When writing, the *options* argument can be a dictionary
-   providing advanced decompression parameters; see
+   providing advanced compression parameters; see
    :class:`CompressionParameter` for detailed information about supported
    parameters. The *level* argument is the compression level to use when
    writing compressed data. Only one of *level* or *options* may be non-None.
@@ -117,7 +117,7 @@ Reading and writing compressed files
    argument is not None, a :exc:`!TypeError` will be raised.
 
    When writing, the *options* argument can be a dictionary
-   providing advanced decompression parameters; see
+   providing advanced compression parameters; see
    :class:`CompressionParameter` for detailed information about supported
    parameters. The *level* argument is the compression level to use when
    writing compressed data. Only one of *level* or *options* may be passed. The

Doc/library/importlib.metadata.rst

@@ -418,6 +418,16 @@ Distributions
    equal, even if they relate to the same installed distribution and
    accordingly have the same attributes.
 
+   .. method:: discover(cls, *, context=None, **kwargs)
+
+      Returns an iterable of :class:`Distribution` instances for all packages.
+
+      The optional argument *context* is a :class:`DistributionFinder.Context`
+      instance, used to modify the search for distributions. Alternatively,
+      *kwargs* may contain keyword arguments for constructing a new
+      :class:`!DistributionFinder.Context`.
+
+
 While the module level API described above is the most common and convenient usage,
 you can get all of that information from the :class:`!Distribution` class.
 :class:`!Distribution` is an abstract object that represents the metadata for
@@ -466,6 +476,61 @@ This metadata finder search defaults to ``sys.path``, but varies slightly in how
 - ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
 - ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
 
+.. class:: DistributionFinder
+
+   A :class:`~importlib.abc.MetaPathFinder` subclass capable of discovering
+   installed distributions.
+
+   Custom providers should implement this interface in order to
+   supply metadata.
+
+    .. class:: Context(**kwargs)
+
+       A :class:`!Context` gives a custom provider a means to
+       solicit additional details from the callers of distribution discovery
+       functions like :func:`distributions` or :meth:`Distribution.discover`
+       beyond :attr:`!.name` and :attr:`!.path` when searching
+       for distributions.
+
+       For example, a provider could expose suites of packages in either a
+       "public" or "private" ``realm``. A caller of distribution discovery
+       functions may wish to query only for distributions in a particular realm
+       and could call ``distributions(realm="private")`` to signal to the
+       custom provider to only include distributions from that
+       realm.
+
+       Each :class:`!DistributionFinder` must expect any parameters and should
+       attempt to honor the canonical parameters defined below when
+       appropriate.
+
+       See the section on :ref:`implementing-custom-providers` for more details.
+
+       .. attribute:: name
+
+          Specific name for which a distribution finder should match.
+
+          A :attr:`!.name` of ``None`` matches all distributions.
+
+       .. attribute:: path
+
+          A property providing the sequence of directory paths that a
+          distribution finder should search.
+
+          Typically refers to Python installed package paths such as
+          "site-packages" directories and defaults to :attr:`sys.path`.
+
+
+.. function:: distributions(**kwargs)
+
+   Returns an iterable of :class:`Distribution` instances for all packages.
+
+   The *kwargs* argument may contain either a keyword argument ``context``, a
+   :class:`DistributionFinder.Context` instance, or pass keyword arguments for
+   constructing a new :class:`!DistributionFinder.Context`. The
+   :class:`!DistributionFinder.Context` is used to modify the search for
+   distributions.
+
+.. _implementing-custom-providers:
 
 Implementing Custom Providers
 =============================
@@ -493,7 +558,7 @@ interface expected of finders by Python's import system.
 ``importlib.metadata`` extends this protocol by looking for an optional
 ``find_distributions`` callable on the finders from
 :data:`sys.meta_path` and presents this extended interface as the
-``DistributionFinder`` abstract base class, which defines this abstract
+:class:`DistributionFinder` abstract base class, which defines this abstract
 method::
 
     @abc.abstractmethod
@@ -502,9 +567,11 @@ method::
         loading the metadata for packages for the indicated ``context``.
         """
 
-The ``DistributionFinder.Context`` object provides ``.path`` and ``.name``
-properties indicating the path to search and name to match and may
-supply other relevant context sought by the consumer.
+The :class:`DistributionFinder.Context` object provides
+:attr:`~DistributionFinder.Context.path` and
+:attr:`~DistributionFinder.Context.name` properties indicating the path to
+search and name to match and may supply other relevant context sought by the
+consumer.
 
 In practice, to support finding distribution package
 metadata in locations other than the file system, subclass
@@ -529,7 +596,7 @@ Imagine a custom finder that loads Python modules from a database::
 That importer now presumably provides importable modules from a
 database, but it provides no metadata or entry points. For this
 custom importer to provide metadata, it would also need to implement
-``DistributionFinder``::
+:class:`DistributionFinder`::
 
     from importlib.metadata import DistributionFinder
 

Doc/library/profiling.sampling.rst

@@ -878,9 +878,9 @@ interesting functions that highlights:
 
 Use :option:`--no-summary` to suppress both the legend and summary sections.
 
-To save pstats output to a file instead of stdout::
+To save pstats output to a binary file instead of stdout::
 
-   python -m profiling.sampling run -o profile.txt script.py
+   python -m profiling.sampling run -o profile.pstats script.py
 
 The pstats format supports several options for controlling the display.
 The :option:`--sort` option determines the column used for ordering results::
@@ -1455,7 +1455,9 @@ Output options
 
 .. option:: --pstats
 
-   Generate text statistics output. This is the default.
+   Generate pstats statistics. This is the default.
+   When written to stdout, the output is a text table; with :option:`-o`,
+   it is a binary pstats file.
 
 .. option:: --collapsed
 
@@ -1486,8 +1488,9 @@ Output options
 .. option:: -o <path>, --output <path>
 
    Output file or directory path. Default behavior varies by format:
-   :option:`--pstats` writes to stdout, while other formats generate a file
-   named ``<format>_<PID>.<ext>`` (for example, ``flamegraph_12345.html``).
+   :option:`--pstats` prints a text table to stdout, while ``-o`` writes a
+   binary pstats file. Other formats generate a file named
+   ``<format>_<PID>.<ext>`` (for example, ``flamegraph_12345.html``).
    :option:`--heatmap` creates a directory named ``heatmap_<PID>``.
 
 .. option:: --browser

Doc/whatsnew/3.15.rst

@@ -1412,7 +1412,7 @@ that may require changes to your code.
   :func:`resource.setrlimit` and :func:`resource.prlimit` is now deprecated.
   (Contributed by Serhiy Storchaka in :gh:`137044`.)
 
-* :meth:`~mmap.mmap.resize` has been removed on platforms that don't support the
+* :meth:`mmap.mmap.resize` has been removed on platforms that don't support the
   underlying syscall, instead of raising a :exc:`SystemError`.
 
 * A resource warning is now emitted for an unclosed

Include/internal/pycore_backoff.h

@@ -12,6 +12,7 @@ extern "C" {
 #include <assert.h>
 #include <stdbool.h>
 #include "pycore_structs.h"       // _Py_BackoffCounter
+#include "pycore_tstate.h"        // _PyPolicy
 
 /* 16-bit countdown counters using exponential backoff.
 
@@ -127,10 +128,11 @@ trigger_backoff_counter(void)
 #define JUMP_BACKWARD_INITIAL_VALUE 4000
 #define JUMP_BACKWARD_INITIAL_BACKOFF 6
 static inline _Py_BackoffCounter
-initial_jump_backoff_counter(void)
+initial_jump_backoff_counter(_PyPolicy *policy)
 {
-    return make_backoff_counter(JUMP_BACKWARD_INITIAL_VALUE,
-                                JUMP_BACKWARD_INITIAL_BACKOFF);
+    return make_backoff_counter(
+        policy->interp.jump_backward_initial_value,
+        policy->interp.jump_backward_initial_backoff);
 }
 
 /* Initial exit temperature.
@@ -141,10 +143,11 @@ initial_jump_backoff_counter(void)
 #define SIDE_EXIT_INITIAL_BACKOFF 6
 
 static inline _Py_BackoffCounter
-initial_temperature_backoff_counter(void)
+initial_temperature_backoff_counter(_PyPolicy *policy)
 {
-    return make_backoff_counter(SIDE_EXIT_INITIAL_VALUE,
-                                SIDE_EXIT_INITIAL_BACKOFF);
+    return make_backoff_counter(
+        policy->jit.side_exit_initial_value,
+        policy->jit.side_exit_initial_backoff);
 }
 
 /* Unreachable backoff counter. */

Include/internal/pycore_magic_number.h

@@ -286,7 +286,8 @@ Known values:
     Python 3.15a1 3653 (Fix handling of opcodes that may leave operands on the stack when optimizing LOAD_FAST)
     Python 3.15a1 3654 (Fix missing exception handlers in logical expression)
     Python 3.15a1 3655 (Fix miscompilation of some module-level annotations)
-    Python 3.15a1 3656 (Add TRACE_RECORD instruction, for platforms with switch based interpreter)
+    Python 3.15a2 3656 (Add TRACE_RECORD instruction, for platforms with switch based interpreter)
+    Python 3.15a4 3657 (Add BINARY_OP_SUBSCR_USTR_INT)
 
 
     Python 3.16 will start with 3700
@@ -300,7 +301,7 @@ PC/launcher.c must also be updated.
 
 */
 
-#define PYC_MAGIC_NUMBER 3656
+#define PYC_MAGIC_NUMBER 3657
 /* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes
    (little-endian) and then appending b'\r\n'. */
 #define PYC_MAGIC_NUMBER_TOKEN \

Include/internal/pycore_opcode_metadata.h

@@ -52,8 +52,24 @@ typedef struct _PyJitTracerState {
     _PyJitTracerInitialState initial_state;
     _PyJitTracerPreviousState prev_state;
 } _PyJitTracerState;
+
 #endif
 
+typedef struct _PyJitPolicy {
+    uint16_t side_exit_initial_value;
+    uint16_t side_exit_initial_backoff;
+} _PyJitPolicy;
+
+typedef struct _PyInterpreterPolicy {
+    uint16_t jump_backward_initial_value;
+    uint16_t jump_backward_initial_backoff;
+} _PyInterpreterPolicy;
+
+typedef struct _PyPolicy {
+    _PyJitPolicy jit;
+    _PyInterpreterPolicy interp;
+} _PyPolicy;
+
 // Every PyThreadState is actually allocated as a _PyThreadStateImpl. The
 // PyThreadState fields are exposed as part of the C API, although most fields
 // are intended to be private. The _PyThreadStateImpl fields not exposed.
@@ -132,6 +148,7 @@ typedef struct _PyThreadStateImpl {
 #if _Py_TIER2
     _PyJitTracerState jit_tracer_state;
 #endif
+    _PyPolicy policy;
 } _PyThreadStateImpl;
 
 #ifdef __cplusplus