Merge branch 'main' into series-input

Author: jialuoo

.pre-commit-config.yaml

@@ -46,4 +46,4 @@ repos:
     rev: v2.0.2
     hooks:
     -   id: biome-check
-        files: '\.js$'
+        files: '\.(js|css)$'

CHANGELOG.md

@@ -4,6 +4,52 @@
 
 [1]: https://pypi.org/project/bigframes/#history
 
+## [2.13.0](https://github.com/googleapis/python-bigquery-dataframes/compare/v2.12.0...v2.13.0) (2025-07-25)
+
+
+### Features
+
+* _read_gbq_colab creates hybrid session ([#1901](https://github.com/googleapis/python-bigquery-dataframes/issues/1901)) ([31b17b0](https://github.com/googleapis/python-bigquery-dataframes/commit/31b17b01706ccfcee9a2d838c43a9609ec4dc218))
+* Add CSS styling for TableWidget pagination interface ([#1934](https://github.com/googleapis/python-bigquery-dataframes/issues/1934)) ([5b232d7](https://github.com/googleapis/python-bigquery-dataframes/commit/5b232d7e33563196316f5dbb50b28c6be388d440))
+* Add row numbering local pushdown in hybrid execution ([#1932](https://github.com/googleapis/python-bigquery-dataframes/issues/1932)) ([92a2377](https://github.com/googleapis/python-bigquery-dataframes/commit/92a237712aa4ce516b1a44748127b34d7780fff6))
+* Implement Index.get_loc ([#1921](https://github.com/googleapis/python-bigquery-dataframes/issues/1921)) ([bbbcaf3](https://github.com/googleapis/python-bigquery-dataframes/commit/bbbcaf35df113617fd6bb8ae36468cf3f7ab493b))
+
+
+### Bug Fixes
+
+* Add license header and correct issues in dbt sample ([#1931](https://github.com/googleapis/python-bigquery-dataframes/issues/1931)) ([ab01b0a](https://github.com/googleapis/python-bigquery-dataframes/commit/ab01b0a236ffc7b667f258e0497105ea5c3d3aab))
+
+
+### Dependencies
+
+* Replace `google-cloud-iam` with `grpc-google-iam-v1` ([#1864](https://github.com/googleapis/python-bigquery-dataframes/issues/1864)) ([e5ff8f7](https://github.com/googleapis/python-bigquery-dataframes/commit/e5ff8f7d9fdac3ea47dabcc80a2598d601f39e64))
+
+## [2.12.0](https://github.com/googleapis/python-bigquery-dataframes/compare/v2.11.0...v2.12.0) (2025-07-23)
+
+
+### Features
+
+* Add code samples for dbt bigframes integration ([#1898](https://github.com/googleapis/python-bigquery-dataframes/issues/1898)) ([7e03252](https://github.com/googleapis/python-bigquery-dataframes/commit/7e03252d31e505731db113eb38af77842bf29b9b))
+* Add isin local execution to hybrid engine ([#1915](https://github.com/googleapis/python-bigquery-dataframes/issues/1915)) ([c0cefd3](https://github.com/googleapis/python-bigquery-dataframes/commit/c0cefd36cfd55962b86178d2a612d625ed17f79c))
+* Add ml.metrics.mean_absolute_error method ([#1910](https://github.com/googleapis/python-bigquery-dataframes/issues/1910)) ([15b8449](https://github.com/googleapis/python-bigquery-dataframes/commit/15b8449dc5ad0c8190a5cbf47894436de18c8e88))
+* Allow local arithmetic execution in hybrid engine ([#1906](https://github.com/googleapis/python-bigquery-dataframes/issues/1906)) ([ebdcd02](https://github.com/googleapis/python-bigquery-dataframes/commit/ebdcd0240f0d8edaef3094b3a4e664b4a84d4a25))
+* Provide day_of_year and day_of_week for dt accessor ([#1911](https://github.com/googleapis/python-bigquery-dataframes/issues/1911)) ([40e7638](https://github.com/googleapis/python-bigquery-dataframes/commit/40e76383948a79bde48108f6180fd6ae2b3d0875))
+* Support params `max_batching_rows`, `container_cpu`, and `container_memory` for `udf` ([#1897](https://github.com/googleapis/python-bigquery-dataframes/issues/1897)) ([8baa912](https://github.com/googleapis/python-bigquery-dataframes/commit/8baa9126e595ae682469a6bb462244240699f57f))
+* Support typed pyarrow.Scalar in assignment  ([#1930](https://github.com/googleapis/python-bigquery-dataframes/issues/1930)) ([cd28e12](https://github.com/googleapis/python-bigquery-dataframes/commit/cd28e12b3f70a6934a68963a7f25dbd5e3c67335))
+
+
+### Bug Fixes
+
+* Correct min field from max() to min() in remote function tests ([#1917](https://github.com/googleapis/python-bigquery-dataframes/issues/1917)) ([d5c54fc](https://github.com/googleapis/python-bigquery-dataframes/commit/d5c54fca32ed75c1aef52c99781db7f8ac7426e1))
+* Resolve location reset issue in bigquery options ([#1914](https://github.com/googleapis/python-bigquery-dataframes/issues/1914)) ([c15cb8a](https://github.com/googleapis/python-bigquery-dataframes/commit/c15cb8a1a9c834c2c1c2984930415b246f3f948b))
+* Series.str.isdigit in unicode superscripts and fractions ([#1924](https://github.com/googleapis/python-bigquery-dataframes/issues/1924)) ([8d46c36](https://github.com/googleapis/python-bigquery-dataframes/commit/8d46c36da7881a99861166c03a0831beff8ee0dd))
+
+
+### Documentation
+
+* Add code snippets for session and IO public docs ([#1919](https://github.com/googleapis/python-bigquery-dataframes/issues/1919)) ([6e01cbe](https://github.com/googleapis/python-bigquery-dataframes/commit/6e01cbec0dcf40e528b4a96e944681df18773c11))
+* Add snippets for performance optimization doc ([#1923](https://github.com/googleapis/python-bigquery-dataframes/issues/1923)) ([4da309e](https://github.com/googleapis/python-bigquery-dataframes/commit/4da309e27bd58a685e8aca953717da75d4ba5305))
+
 ## [2.11.0](https://github.com/googleapis/python-bigquery-dataframes/compare/v2.10.0...v2.11.0) (2025-07-15)
 
 

MANIFEST.in

@@ -17,7 +17,7 @@
 # Generated by synthtool. DO NOT EDIT!
 include README.rst LICENSE
 recursive-include third_party/bigframes_vendored *
-recursive-include bigframes *.json *.proto *.js py.typed
+recursive-include bigframes *.json *.proto *.js *.css py.typed
 recursive-include tests *
 global-exclude *.py[co]
 global-exclude __pycache__

bigframes/core/compile/polars/compiler.py

@@ -513,6 +513,30 @@ def compile_join(self, node: nodes.JoinNode):
             left, right, node.type, left_on, right_on, node.joins_nulls
         )
 
+    @compile_node.register
+    def compile_isin(self, node: nodes.InNode):
+        left = self.compile_node(node.left_child)
+        right = self.compile_node(node.right_child).unique(node.right_col.id.sql)
+        right = right.with_columns(pl.lit(True).alias(node.indicator_col.sql))
+
+        left_ex, right_ex = lowering._coerce_comparables(node.left_col, node.right_col)
+
+        left_pl_ex = self.expr_compiler.compile_expression(left_ex)
+        right_pl_ex = self.expr_compiler.compile_expression(right_ex)
+
+        joined = left.join(
+            right,
+            how="left",
+            left_on=left_pl_ex,
+            right_on=right_pl_ex,
+            # Note: join_nulls renamed to nulls_equal for polars 1.24
+            join_nulls=node.joins_nulls,  # type: ignore
+            coalesce=False,
+        )
+        passthrough = [pl.col(id) for id in left.columns]
+        indicator = pl.col(node.indicator_col.sql).fill_null(False)
+        return joined.select((*passthrough, indicator))
+
     def _ordered_join(
         self,
         left_frame: pl.LazyFrame,

bigframes/core/compile/scalar_op_compiler.py

@@ -487,9 +487,9 @@ def isalpha_op_impl(x: ibis_types.Value):
 
 @scalar_op_compiler.register_unary_op(ops.isdigit_op)
 def isdigit_op_impl(x: ibis_types.Value):
-    # Based on docs, should include superscript/subscript-ed numbers
-    # Tests however pass only when set to Nd unicode class
-    return typing.cast(ibis_types.StringValue, x).re_search(r"^(\p{Nd})+$")
+    return typing.cast(ibis_types.StringValue, x).re_search(
+        r"^[\p{Nd}\x{00B9}\x{00B2}\x{00B3}\x{2070}\x{2074}-\x{2079}\x{2080}-\x{2089}]+$"
+    )
 
 
 @scalar_op_compiler.register_unary_op(ops.isdecimal_op)

bigframes/core/compile/sqlglot/aggregate_compiler.py

@@ -13,16 +13,17 @@
 # limitations under the License.
 from __future__ import annotations
 
-import functools
-import typing
-
 import sqlglot.expressions as sge
 
-from bigframes.core import expression, window_spec
+from bigframes.core import expression
+from bigframes.core.compile.sqlglot.aggregations import (
+    binary_compiler,
+    nullary_compiler,
+    ordered_unary_compiler,
+    unary_compiler,
+)
 from bigframes.core.compile.sqlglot.expressions import typed_expr
 import bigframes.core.compile.sqlglot.scalar_compiler as scalar_compiler
-import bigframes.core.compile.sqlglot.sqlglot_ir as ir
-import bigframes.operations as ops
 
 
 def compile_aggregate(
@@ -31,16 +32,18 @@ def compile_aggregate(
 ) -> sge.Expression:
     """Compiles BigFrames aggregation expression into SQLGlot expression."""
     if isinstance(aggregate, expression.NullaryAggregation):
-        return compile_nullary_agg(aggregate.op)
+        return nullary_compiler.compile(aggregate.op)
     if isinstance(aggregate, expression.UnaryAggregation):
         column = typed_expr.TypedExpr(
             scalar_compiler.compile_scalar_expression(aggregate.arg),
             aggregate.arg.output_type,
         )
         if not aggregate.op.order_independent:
-            return compile_ordered_unary_agg(aggregate.op, column, order_by=order_by)
+            return ordered_unary_compiler.compile(
+                aggregate.op, column, order_by=order_by
+            )
         else:
-            return compile_unary_agg(aggregate.op, column)
+            return unary_compiler.compile(aggregate.op, column)
     elif isinstance(aggregate, expression.BinaryAggregation):
         left = typed_expr.TypedExpr(
             scalar_compiler.compile_scalar_expression(aggregate.left),
@@ -50,63 +53,6 @@ def compile_aggregate(
             scalar_compiler.compile_scalar_expression(aggregate.right),
             aggregate.right.output_type,
         )
-        return compile_binary_agg(aggregate.op, left, right)
+        return binary_compiler.compile(aggregate.op, left, right)
     else:
         raise ValueError(f"Unexpected aggregation: {aggregate}")
-
-
-@functools.singledispatch
-def compile_nullary_agg(
-    op: ops.aggregations.WindowOp,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-) -> sge.Expression:
-    raise ValueError(f"Can't compile unrecognized operation: {op}")
-
-
-@functools.singledispatch
-def compile_binary_agg(
-    op: ops.aggregations.WindowOp,
-    left: typed_expr.TypedExpr,
-    right: typed_expr.TypedExpr,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-) -> sge.Expression:
-    raise ValueError(f"Can't compile unrecognized operation: {op}")
-
-
-@functools.singledispatch
-def compile_unary_agg(
-    op: ops.aggregations.WindowOp,
-    column: typed_expr.TypedExpr,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-) -> sge.Expression:
-    raise ValueError(f"Can't compile unrecognized operation: {op}")
-
-
-@functools.singledispatch
-def compile_ordered_unary_agg(
-    op: ops.aggregations.WindowOp,
-    column: typed_expr.TypedExpr,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-    order_by: typing.Sequence[sge.Expression] = [],
-) -> sge.Expression:
-    raise ValueError(f"Can't compile unrecognized operation: {op}")
-
-
-@compile_unary_agg.register
-def _(
-    op: ops.aggregations.SumOp,
-    column: typed_expr.TypedExpr,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-) -> sge.Expression:
-    # Will be null if all inputs are null. Pandas defaults to zero sum though.
-    expr = _apply_window_if_present(sge.func("SUM", column.expr), window)
-    return sge.func("IFNULL", expr, ir._literal(0, column.dtype))
-
-
-def _apply_window_if_present(
-    value: sge.Expression,
-    window: typing.Optional[window_spec.WindowSpec] = None,
-) -> sge.Expression:
-    if window is not None:
-        raise NotImplementedError("Can't apply window to the expression.")
-    return value

bigframes/core/compile/sqlglot/aggregations/__init__.py

@@ -0,0 +1,13 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

bigframes/core/compile/sqlglot/aggregations/binary_compiler.py

@@ -0,0 +1,35 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+import typing
+
+import sqlglot.expressions as sge
+
+from bigframes.core import window_spec
+import bigframes.core.compile.sqlglot.aggregations.op_registration as reg
+import bigframes.core.compile.sqlglot.expressions.typed_expr as typed_expr
+from bigframes.operations import aggregations as agg_ops
+
+BINARY_OP_REGISTRATION = reg.OpRegistration()
+
+
+def compile(
+    op: agg_ops.WindowOp,
+    left: typed_expr.TypedExpr,
+    right: typed_expr.TypedExpr,
+    window: typing.Optional[window_spec.WindowSpec] = None,
+) -> sge.Expression:
+    return BINARY_OP_REGISTRATION[op](op, left, right, window=window)

bigframes/core/compile/sqlglot/aggregations/nullary_compiler.py

@@ -0,0 +1,41 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+import typing
+
+import sqlglot.expressions as sge
+
+from bigframes.core import window_spec
+import bigframes.core.compile.sqlglot.aggregations.op_registration as reg
+from bigframes.core.compile.sqlglot.aggregations.utils import apply_window_if_present
+from bigframes.operations import aggregations as agg_ops
+
+NULLARY_OP_REGISTRATION = reg.OpRegistration()
+
+
+def compile(
+    op: agg_ops.WindowOp,
+    window: typing.Optional[window_spec.WindowSpec] = None,
+) -> sge.Expression:
+    return NULLARY_OP_REGISTRATION[op](op, window=window)
+
+
+@NULLARY_OP_REGISTRATION.register(agg_ops.SizeOp)
+def _(
+    op: agg_ops.SizeOp,
+    window: typing.Optional[window_spec.WindowSpec] = None,
+) -> sge.Expression:
+    return apply_window_if_present(sge.func("COUNT", sge.convert(1)), window)

bigframes/core/compile/sqlglot/aggregations/op_registration.py

@@ -0,0 +1,62 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+import typing
+
+from sqlglot import expressions as sge
+
+from bigframes.operations import aggregations as agg_ops
+
+# We should've been more specific about input types. Unfortunately,
+# MyPy doesn't support more rigorous checks.
+CompilationFunc = typing.Callable[..., sge.Expression]
+
+
+class OpRegistration:
+    def __init__(self) -> None:
+        self._registered_ops: dict[str, CompilationFunc] = {}
+
+    def register(
+        self, op: agg_ops.WindowOp | type[agg_ops.WindowOp]
+    ) -> typing.Callable[[CompilationFunc], CompilationFunc]:
+        def decorator(item: CompilationFunc):
+            def arg_checker(*args, **kwargs):
+                if not isinstance(args[0], agg_ops.WindowOp):
+                    raise ValueError(
+                        "The first parameter must be a window operator. "
+                        f"Got {type(args[0])}"
+                    )
+                return item(*args, **kwargs)
+
+            if hasattr(op, "name"):
+                key = typing.cast(str, op.name)
+                if key in self._registered_ops:
+                    raise ValueError(f"{key} is already registered")
+            else:
+                raise ValueError(f"The operator must have a 'name' attribute. Got {op}")
+            self._registered_ops[key] = item
+            return arg_checker
+
+        return decorator
+
+    def __getitem__(self, op: str | agg_ops.WindowOp) -> CompilationFunc:
+        if isinstance(op, agg_ops.WindowOp):
+            if not hasattr(op, "name"):
+                raise ValueError(f"The operator must have a 'name' attribute. Got {op}")
+            else:
+                key = typing.cast(str, op.name)
+                return self._registered_ops[key]
+        return self._registered_ops[op]