From a637bde872f60283e322936883e1d7dd61394e77 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 09:46:33 +0200 Subject: [PATCH 1/9] gh-137205: Document how to safely use PRAGMA during SQLite transactions --- Doc/library/sqlite3.rst | 44 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index d317ead66f9bcb..b4160c0aa31263 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -243,6 +243,7 @@ inserted data and retrieved values from it in multiple ways. * :ref:`sqlite3-converters` * :ref:`sqlite3-connection-context-manager` * :ref:`sqlite3-howto-row-factory` + * :ref:`sqlite3-howto-pragma-in-transaction` * :ref:`sqlite3-explanation` for in-depth background on transaction control. @@ -1353,8 +1354,6 @@ Connection objects implying that :mod:`!sqlite3` ensures a transaction is always open. Use :meth:`commit` and :meth:`rollback` to close transactions. - This is the recommended value of :attr:`!autocommit`. - * ``True``: Use SQLite's `autocommit mode`_. :meth:`commit` and :meth:`rollback` have no effect in this mode. @@ -2429,6 +2428,34 @@ the context manager does nothing. couldn't add Python twice +.. _sqlite3-howto-pragma-in-transaction: + +How to use ``PRAGMA`` statements in transactions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Some ``PRAGMA`` statements have no effect within transactions. +For example, a ``PRAGMA foreign_keys=ON`` query will silently fail +if there is an open transaction. +In order to safely use ``PRAGMA`` statements, +ensure that there is no open transaction: + +* If :attr:`~Connection.autocommit` is ``True`` + and a transaction was explicitly opened, + make sure to :meth:`~Cursor.execute` a ``COMMIT`` + before executing the ``PRAGMA`` statement. +* Else, temporarily disable implicit transaction control + using the :attr:`!autocommit` attribute + before executing the ``PRAGMA`` statement: + +.. testcode:: + + saved = cur.autocommit + conn.autocommit = True # Disable implicit transaction control. + cursor.execute("PRAGMA foreign_keys=ON") + conn.autocommit = saved # Restore the previous setting. + + + .. _sqlite3-uri-tricks: How to work with SQLite URIs @@ -2646,8 +2673,7 @@ the :attr:`Connection.autocommit` attribute, which should preferably be set using the *autocommit* parameter of :func:`connect`. -It is suggested to set *autocommit* to ``False``, -which implies :pep:`249`-compliant transaction control. +Set *autocommit* to ``False`` for :pep:`249`-compliant transaction control. This means: * :mod:`!sqlite3` ensures that a transaction is always open, @@ -2660,6 +2686,11 @@ This means: * An implicit rollback is performed if the database is :meth:`~Connection.close`-ed with pending changes. +.. note:: + + Some ``PRAGMA`` statements cannot be set when a transaction is open. + See :ref:`sqlite3-howto-pragma-in-transaction` for details. + Set *autocommit* to ``True`` to enable SQLite's `autocommit mode`_. In this mode, :meth:`Connection.commit` and :meth:`Connection.rollback` have no effect. @@ -2716,6 +2747,11 @@ The :meth:`~Cursor.executescript` method implicitly commits any pending transaction before execution of the given SQL script, regardless of the value of :attr:`~Connection.isolation_level`. +.. note:: + + Some ``PRAGMA`` statements cannot be set when a transaction is open. + See :ref:`sqlite3-howto-pragma-in-transaction` for details. + .. versionchanged:: 3.6 :mod:`!sqlite3` used to implicitly commit an open transaction before DDL statements. This is no longer the case. From 9b0e56e44bc6f8f2a523018aaae5e93dd5ec91be Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 09:52:13 +0200 Subject: [PATCH 2/9] conn => con --- Doc/library/sqlite3.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index b4160c0aa31263..6ad17d2808a4d5 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2450,9 +2450,9 @@ ensure that there is no open transaction: .. testcode:: saved = cur.autocommit - conn.autocommit = True # Disable implicit transaction control. + con.autocommit = True # Disable implicit transaction control. cursor.execute("PRAGMA foreign_keys=ON") - conn.autocommit = saved # Restore the previous setting. + con.autocommit = saved # Restore the previous setting. From b525dd86ce93bc3a34c8ede3bb0e04fc2b2d3918 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 09:56:29 +0200 Subject: [PATCH 3/9] Whitespace --- Doc/library/sqlite3.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 6ad17d2808a4d5..4f23c2e6485447 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2455,7 +2455,6 @@ ensure that there is no open transaction: con.autocommit = saved # Restore the previous setting. - .. _sqlite3-uri-tricks: How to work with SQLite URIs From ae5d70028358af765e48a48076d800c067f77dc0 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 10:05:19 +0200 Subject: [PATCH 4/9] Promote the new howto in the autocommit attr reference --- Doc/library/sqlite3.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 4f23c2e6485447..ec49c84cf03f02 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1366,7 +1366,10 @@ Connection objects Changing :attr:`!autocommit` to ``False`` will open a new transaction, and changing it to ``True`` will commit any pending transaction. - See :ref:`sqlite3-transaction-control-autocommit` for more details. + .. seealso: + + * :ref:`sqlite3-transaction-control-autocommit` + * :ref:`sqlite3-howto-pragma-in-transaction` .. note:: @@ -2687,7 +2690,7 @@ This means: .. note:: - Some ``PRAGMA`` statements cannot be set when a transaction is open. + Some ``PRAGMA`` statements have no effect when a transaction is open. See :ref:`sqlite3-howto-pragma-in-transaction` for details. Set *autocommit* to ``True`` to enable SQLite's `autocommit mode`_. From fc2e9570561e094d9233df54ec17106b69b6de1a Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 10:06:07 +0200 Subject: [PATCH 5/9] Fix doctest --- Doc/library/sqlite3.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index ec49c84cf03f02..5554dc71934ad0 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2452,9 +2452,9 @@ ensure that there is no open transaction: .. testcode:: - saved = cur.autocommit + saved = con.autocommit con.autocommit = True # Disable implicit transaction control. - cursor.execute("PRAGMA foreign_keys=ON") + cur.execute("PRAGMA foreign_keys=ON") con.autocommit = saved # Restore the previous setting. From 5ef2d72e258621c928e12537f272c5d7bcd75f37 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 10:08:42 +0200 Subject: [PATCH 6/9] Add missing semicolon to seealso Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> --- Doc/library/sqlite3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 5554dc71934ad0..bce86ff413a2b9 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1366,7 +1366,7 @@ Connection objects Changing :attr:`!autocommit` to ``False`` will open a new transaction, and changing it to ``True`` will commit any pending transaction. - .. seealso: + .. seealso:: * :ref:`sqlite3-transaction-control-autocommit` * :ref:`sqlite3-howto-pragma-in-transaction` From 155013f929ec2e97e60bf275b3c106f6ba38532f Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 7 Aug 2025 11:00:17 +0200 Subject: [PATCH 7/9] Correctly setup and format example doctest code --- Doc/library/sqlite3.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index bce86ff413a2b9..3a8bdd0c83af88 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2450,12 +2450,18 @@ ensure that there is no open transaction: using the :attr:`!autocommit` attribute before executing the ``PRAGMA`` statement: -.. testcode:: + .. testcode:: + :hide: + + con = sqlite3.connect(":memory:", autocommit=False) + + .. testcode:: - saved = con.autocommit - con.autocommit = True # Disable implicit transaction control. - cur.execute("PRAGMA foreign_keys=ON") - con.autocommit = saved # Restore the previous setting. + cur = con.cursor() + saved = con.autocommit + con.autocommit = True # Disable implicit transaction control. + cur.execute("PRAGMA foreign_keys=ON") + con.autocommit = saved # Restore the previous setting. .. _sqlite3-uri-tricks: From 345043fcd2c13d0bdcf29e4bbd95c6cf90c9c705 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Mon, 11 Aug 2025 10:55:09 +0200 Subject: [PATCH 8/9] Amend How-to heading, and remove a recommendation --- Doc/library/sqlite3.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 3a8bdd0c83af88..c544c16f0cf176 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2433,8 +2433,8 @@ the context manager does nothing. .. _sqlite3-howto-pragma-in-transaction: -How to use ``PRAGMA`` statements in transactions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +How to use ``PRAGMA`` statements with implicit transaction control +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Some ``PRAGMA`` statements have no effect within transactions. For example, a ``PRAGMA foreign_keys=ON`` query will silently fail @@ -2678,8 +2678,7 @@ Transaction control via the ``autocommit`` attribute The recommended way of controlling transaction behaviour is through the :attr:`Connection.autocommit` attribute, -which should preferably be set using the *autocommit* parameter -of :func:`connect`. +normally set using the *autocommit* parameter of :func:`connect`. Set *autocommit* to ``False`` for :pep:`249`-compliant transaction control. This means: From 000e4c5d9f74635e5fd5aa25071697c914b402b5 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Mon, 11 Aug 2025 11:04:29 +0200 Subject: [PATCH 9/9] Mention layday's suggestion --- Doc/library/sqlite3.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index c544c16f0cf176..123f18e822100e 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2696,6 +2696,11 @@ This means: .. note:: Some ``PRAGMA`` statements have no effect when a transaction is open. + If ``PRAGMA`` statements are a part of the database connection setup + and you want :pep:`249`-compliant transaction control, + set *autocommit* to ``True`` in :func:`connect`, + execute the needed ``PRAGMA`` statements, + then set :attr:`Connection.autocommit` to ``False``. See :ref:`sqlite3-howto-pragma-in-transaction` for details. Set *autocommit* to ``True`` to enable SQLite's `autocommit mode`_.