Merge branch 'main' into feature/migrate-strconcat-op

Author: jialuoo

CHANGELOG.md

@@ -4,6 +4,34 @@
 
 [1]: https://pypi.org/project/bigframes/#history
 
+## [2.25.0](https://github.com/googleapis/python-bigquery-dataframes/compare/v2.24.0...v2.25.0) (2025-10-13)
+
+
+### Features
+
+* Add barh, pie plot types ([#2146](https://github.com/googleapis/python-bigquery-dataframes/issues/2146)) ([5cc3c5b](https://github.com/googleapis/python-bigquery-dataframes/commit/5cc3c5b1391a7dfa062b1d77f001726b013f6337))
+* Add Index.__eq__ for consts, aligned objects ([#2141](https://github.com/googleapis/python-bigquery-dataframes/issues/2141)) ([8514200](https://github.com/googleapis/python-bigquery-dataframes/commit/85142008ec895fa078d192bbab942d0257f70df3))
+* Add output_schema parameter to ai.generate() ([#2139](https://github.com/googleapis/python-bigquery-dataframes/issues/2139)) ([ef0b0b7](https://github.com/googleapis/python-bigquery-dataframes/commit/ef0b0b73843da2a93baf08e4cd5457fbb590b89c))
+* Create session-scoped `cut`, `DataFrame`, `MultiIndex`, `Index`, `Series`, `to_datetime`, and `to_timedelta` methods ([#2157](https://github.com/googleapis/python-bigquery-dataframes/issues/2157)) ([5e1e809](https://github.com/googleapis/python-bigquery-dataframes/commit/5e1e8098ecf212c91d73fa80d722d1cb3e46668b))
+* Replace ML.GENERATE_TEXT with AI.GENERATE for audio transcription ([#2151](https://github.com/googleapis/python-bigquery-dataframes/issues/2151)) ([a410d0a](https://github.com/googleapis/python-bigquery-dataframes/commit/a410d0ae43ef3b053b650804156eda0b1f569da9))
+* Support string literal inputs for AI functions ([#2152](https://github.com/googleapis/python-bigquery-dataframes/issues/2152)) ([7600001](https://github.com/googleapis/python-bigquery-dataframes/commit/760000122dc190ac8a3303234cf4cbee1bbb9493))
+
+
+### Bug Fixes
+
+* Address typo in error message ([#2142](https://github.com/googleapis/python-bigquery-dataframes/issues/2142)) ([cdf2dd5](https://github.com/googleapis/python-bigquery-dataframes/commit/cdf2dd55a0c03da50ab92de09788cafac0abf6f6))
+* Avoid possible circular imports in global session ([#2115](https://github.com/googleapis/python-bigquery-dataframes/issues/2115)) ([095c0b8](https://github.com/googleapis/python-bigquery-dataframes/commit/095c0b85a25a2e51087880909597cc62a0341c93))
+* Fix too many cluster columns requested by caching ([#2155](https://github.com/googleapis/python-bigquery-dataframes/issues/2155)) ([35c1c33](https://github.com/googleapis/python-bigquery-dataframes/commit/35c1c33b85d1b92e402aab73677df3ffe43a51b4))
+* Show progress even in job optional queries ([#2119](https://github.com/googleapis/python-bigquery-dataframes/issues/2119)) ([1f48d3a](https://github.com/googleapis/python-bigquery-dataframes/commit/1f48d3a62e7e6dac4acb39e911daf766b8e2fe62))
+* Yield row count from read session if otherwise unknown ([#2148](https://github.com/googleapis/python-bigquery-dataframes/issues/2148)) ([8997d4d](https://github.com/googleapis/python-bigquery-dataframes/commit/8997d4d7d9965e473195f98c550c80657035b7e1))
+
+
+### Documentation
+
+* Add a brief intro notebook for bbq AI functions ([#2150](https://github.com/googleapis/python-bigquery-dataframes/issues/2150)) ([1f434fb](https://github.com/googleapis/python-bigquery-dataframes/commit/1f434fb5c7c00601654b3ab19c6ad7fceb258bd6))
+* Fix ai function related docs ([#2149](https://github.com/googleapis/python-bigquery-dataframes/issues/2149)) ([93a0749](https://github.com/googleapis/python-bigquery-dataframes/commit/93a0749392b84f27162654fe5ea5baa329a23f99))
+* Remove progress bar from getting started template ([#2143](https://github.com/googleapis/python-bigquery-dataframes/issues/2143)) ([d13abad](https://github.com/googleapis/python-bigquery-dataframes/commit/d13abadbcd68d03997e8dc11bb7a2b14bbd57fcc))
+
 ## [2.24.0](https://github.com/googleapis/python-bigquery-dataframes/compare/v2.23.0...v2.24.0) (2025-10-07)
 
 

bigframes/bigquery/_operations/ai.py

@@ -65,14 +65,21 @@ def generate(
         1    Ottawa\\n
         Name: result, dtype: string
 
-        You get structured output when the `output_schema` parameter is set:
+    You get structured output when the `output_schema` parameter is set:
 
         >>> animals = bpd.Series(["Rabbit", "Spider"])
         >>> bbq.ai.generate(animals, output_schema={"number_of_legs": "INT64", "is_herbivore": "BOOL"})
         0    {'is_herbivore': True, 'number_of_legs': 4, 'f...
         1    {'is_herbivore': False, 'number_of_legs': 8, '...
         dtype: struct<is_herbivore: bool, number_of_legs: int64, full_response: extension<dbjson<JSONArrowType>>, status: string>[pyarrow]
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series
@@ -165,6 +172,13 @@ def generate_bool(
         2    False
         Name: result, dtype: boolean
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series
@@ -240,6 +254,13 @@ def generate_int(
         2    8
         Name: result, dtype: Int64
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series
@@ -315,6 +336,13 @@ def generate_double(
         2    8.0
         Name: result, dtype: Float64
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series
@@ -371,6 +399,7 @@ def if_(
     provides optimization such that not all rows are evaluated with the LLM.
 
     **Examples:**
+
         >>> import bigframes.pandas as bpd
         >>> import bigframes.bigquery as bbq
         >>> bpd.options.display.progress_bar = None
@@ -386,6 +415,13 @@ def if_(
         1         Illinois
         dtype: string
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series
@@ -433,6 +469,13 @@ def classify(
         <BLANKLINE>
         [2 rows x 2 columns]
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         input (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the input to send to the model. The Series can be BigFrames Series
@@ -482,6 +525,13 @@ def score(
         2    3.0
         dtype: Float64
 
+    .. note::
+
+        This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the
+        Service Specific Terms(https://cloud.google.com/terms/service-terms#1). Pre-GA products and features are available "as is"
+        and might have limited support. For more information, see the launch stage descriptions
+        (https://cloud.google.com/products#product-launch-stages).
+
     Args:
         prompt (str | Series | List[str|Series] | Tuple[str|Series, ...]):
             A mixture of Series and string literals that specifies the prompt to send to the model. The Series can be BigFrames Series

bigframes/core/compile/sqlglot/aggregations/ordered_unary_compiler.py

@@ -14,11 +14,8 @@
 
 from __future__ import annotations
 
-import typing
-
 import sqlglot.expressions as sge
 
-from bigframes.core import window_spec
 import bigframes.core.compile.sqlglot.aggregations.op_registration as reg
 import bigframes.core.compile.sqlglot.expressions.typed_expr as typed_expr
 from bigframes.operations import aggregations as agg_ops
@@ -29,9 +26,35 @@
 def compile(
     op: agg_ops.WindowOp,
     column: typed_expr.TypedExpr,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-    order_by: typing.Sequence[sge.Expression] = [],
+    *,
+    order_by: tuple[sge.Expression, ...],
+) -> sge.Expression:
+    return ORDERED_UNARY_OP_REGISTRATION[op](op, column, order_by=order_by)
+
+
+@ORDERED_UNARY_OP_REGISTRATION.register(agg_ops.ArrayAggOp)
+def _(
+    op: agg_ops.ArrayAggOp,
+    column: typed_expr.TypedExpr,
+    *,
+    order_by: tuple[sge.Expression, ...],
 ) -> sge.Expression:
-    return ORDERED_UNARY_OP_REGISTRATION[op](
-        op, column, window=window, order_by=order_by
-    )
+    expr = column.expr
+    if len(order_by) > 0:
+        expr = sge.Order(this=column.expr, expressions=list(order_by))
+    return sge.IgnoreNulls(this=sge.ArrayAgg(this=expr))
+
+
+@ORDERED_UNARY_OP_REGISTRATION.register(agg_ops.StringAggOp)
+def _(
+    op: agg_ops.StringAggOp,
+    column: typed_expr.TypedExpr,
+    *,
+    order_by: tuple[sge.Expression, ...],
+) -> sge.Expression:
+    expr = column.expr
+    if len(order_by) > 0:
+        expr = sge.Order(this=expr, expressions=list(order_by))
+
+    expr = sge.GroupConcat(this=expr, separator=sge.convert(op.sep))
+    return sge.func("COALESCE", expr, sge.convert(""))

bigframes/version.py

@@ -12,8 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "2.24.0"
+__version__ = "2.25.0"
 
 # {x-release-please-start-date}
-__release_date__ = "2025-10-07"
+__release_date__ = "2025-10-13"
 # {x-release-please-end}

docs/templates/toc.yml

@@ -219,7 +219,8 @@
     - name: BigQuery built-in functions
       uid: bigframes.bigquery
     - name: BigQuery AI Functions
-      uid: bigframes.bigquery.ai
+      uid: bigframes.bigquery._operations.ai
+      status: beta
     name: bigframes.bigquery
   - items:
     - name: GeoSeries

tests/system/large/blob/test_function.py

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import logging
 import os
 import traceback
 from typing import Generator
@@ -434,6 +435,15 @@ def test_blob_transcribe(
         actual_text = actual[0]["content"]
     else:
         actual_text = actual[0]
+
+    if pd.isna(actual_text) or actual_text == "":
+        # Ensure the tests are robust to flakes in the model, which isn't
+        # particularly useful information for the bigframes team.
+        logging.warning(
+            f"blob_transcribe() model {model_name} verbose={verbose} failure"
+        )
+        return
+
     actual_len = len(actual_text)
 
     relative_length_tolerance = 0.2

tests/unit/core/compile/sqlglot/aggregations/snapshots/test_ordered_unary_compiler/test_array_agg/out.sql

@@ -0,0 +1,12 @@
+WITH `bfcte_0` AS (
+  SELECT
+    `int64_col` AS `bfcol_0`
+  FROM `bigframes-dev`.`sqlglot_test`.`scalar_types`
+), `bfcte_1` AS (
+  SELECT
+    ARRAY_AGG(`bfcol_0` IGNORE NULLS ORDER BY `bfcol_0` IS NULL ASC, `bfcol_0` ASC) AS `bfcol_1`
+  FROM `bfcte_0`
+)
+SELECT
+  `bfcol_1` AS `int64_col`
+FROM `bfcte_1`

tests/unit/core/compile/sqlglot/aggregations/snapshots/test_ordered_unary_compiler/test_string_agg/out.sql

@@ -0,0 +1,15 @@
+WITH `bfcte_0` AS (
+  SELECT
+    `string_col` AS `bfcol_0`
+  FROM `bigframes-dev`.`sqlglot_test`.`scalar_types`
+), `bfcte_1` AS (
+  SELECT
+    COALESCE(STRING_AGG(`bfcol_0`, ','
+    ORDER BY
+      `bfcol_0` IS NULL ASC,
+      `bfcol_0` ASC), '') AS `bfcol_1`
+  FROM `bfcte_0`
+)
+SELECT
+  `bfcol_1` AS `string_col`
+FROM `bfcte_1`

tests/unit/core/compile/sqlglot/aggregations/test_ordered_unary_compiler.py

@@ -0,0 +1,80 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+import typing
+
+import pytest
+
+from bigframes.core import agg_expressions as agg_exprs
+from bigframes.core import array_value, identifiers, nodes, ordering
+from bigframes.operations import aggregations as agg_ops
+import bigframes.pandas as bpd
+
+pytest.importorskip("pytest_snapshot")
+
+
+def _apply_ordered_unary_agg_ops(
+    obj: bpd.DataFrame,
+    ops_list: typing.Sequence[agg_exprs.UnaryAggregation],
+    new_names: typing.Sequence[str],
+    ordering_args: typing.Sequence[str],
+) -> str:
+    ordering_exprs = tuple(ordering.ascending_over(arg) for arg in ordering_args)
+    aggs = [(op, identifiers.ColumnId(name)) for op, name in zip(ops_list, new_names)]
+
+    agg_node = nodes.AggregateNode(
+        obj._block.expr.node,
+        aggregations=tuple(aggs),
+        by_column_ids=(),
+        order_by=ordering_exprs,
+    )
+    result = array_value.ArrayValue(agg_node)
+
+    sql = result.session._executor.to_sql(result, enable_cache=False)
+    return sql
+
+
+def test_array_agg(scalar_types_df: bpd.DataFrame, snapshot):
+    # TODO: Verify "NULL LAST" syntax issue on Python < 3.12
+    if sys.version_info < (3, 12):
+        pytest.skip(
+            "Skipping test due to inconsistent SQL formatting on Python < 3.12.",
+        )
+
+    col_name = "int64_col"
+    bf_df = scalar_types_df[[col_name]]
+    agg_expr = agg_ops.ArrayAggOp().as_expr(col_name)
+    sql = _apply_ordered_unary_agg_ops(
+        bf_df, [agg_expr], [col_name], ordering_args=[col_name]
+    )
+
+    snapshot.assert_match(sql, "out.sql")
+
+
+def test_string_agg(scalar_types_df: bpd.DataFrame, snapshot):
+    # TODO: Verify "NULL LAST" syntax issue on Python < 3.12
+    if sys.version_info < (3, 12):
+        pytest.skip(
+            "Skipping test due to inconsistent SQL formatting on Python < 3.12.",
+        )
+
+    col_name = "string_col"
+    bf_df = scalar_types_df[[col_name]]
+    agg_expr = agg_ops.StringAggOp(sep=",").as_expr(col_name)
+    sql = _apply_ordered_unary_agg_ops(
+        bf_df, [agg_expr], [col_name], ordering_args=[col_name]
+    )
+
+    snapshot.assert_match(sql, "out.sql")

third_party/bigframes_vendored/version.py

@@ -12,8 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "2.24.0"
+__version__ = "2.25.0"
 
 # {x-release-please-start-date}
-__release_date__ = "2025-10-07"
+__release_date__ = "2025-10-13"
 # {x-release-please-end}