Skip to content

Commit f16c5cb

Browse files
committed
docs: Elaborate on refresh_key with every
1 parent fb5a40f commit f16c5cb

File tree

2 files changed

+36
-6
lines changed

2 files changed

+36
-6
lines changed

docs/pages/product/data-modeling/reference/cube.mdx

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -474,6 +474,21 @@ cubes:
474474
475475
</CodeTabs>
476476
477+
<WarningBox>
478+
479+
The `every` parameter guarantees that the refresh key will be executed **at
480+
least** once during the specified interval. However, it does **not** guarantee
481+
that it will be executed **at most** once. The refresh key may be checked more
482+
frequently.
483+
484+
When using `every` with `sql`, the purpose is to reduce the load from running
485+
the refresh key query itself, not to minimize the number of pre-aggregation
486+
refreshes. The refresh key SQL should be designed to return consistent results
487+
and only change when the underlying data actually changes, ensuring minimal
488+
refreshes even if the query is executed multiple times.
489+
490+
</WarningBox>
491+
477492
`every` - can be set as an interval with granularities `second`, `minute`,
478493
`hour`, `day`, and `week` or accept CRON string with some limitations. If you
479494
set `every` as CRON string, you can use the `timezone` parameter. It takes

docs/pages/product/data-modeling/reference/pre-aggregations.mdx

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ cubes:
5151
- CUBE.status
5252
measures:
5353
- CUBE.count
54-
54+
5555
# ...
5656
```
5757

@@ -809,15 +809,15 @@ tenants in case different tenants have different pre-aggregation SQL.
809809
Choose the count of partitions wisely as those consume memory and CPU resources.
810810
As a rule of thumb, you do not want to go over 500-1,000 partitions per pre-aggregation in total
811811
to keep the partitioning overhead low. Too many partitions will most likely
812-
cause out of memory.
812+
cause out of memory.
813813
In case of very long build ranges please consider use [Lambda pre-aggregations][ref-caching-lambda-preaggs] to reduce partition count per pre-aggregation.
814814

815815
</WarningBox>
816816

817817
### `refresh_key`
818818

819819
Cube can also take care of keeping pre-aggregations up to date with the
820-
`refresh_key` property. By default, it is set to `every: '1 hour'`,
820+
`refresh_key` property. By default, it is set to `every: '1 hour'`,
821821
if neither of the cubes' pre-aggregation references don't override `refresh_key`.
822822

823823
<InfoBox>
@@ -964,6 +964,21 @@ In the above example, the refresh key SQL will be executed every hour. If the
964964
results of the SQL refresh key differ from the last execution, then the
965965
pre-aggregation will be refreshed.
966966
967+
<WarningBox>
968+
969+
The `every` parameter guarantees that the refresh key will be executed **at
970+
least** once during the specified interval. However, it does **not** guarantee
971+
that it will be executed **at most** once. The refresh key may be checked more
972+
frequently.
973+
974+
When using `every` with `sql`, the purpose is to reduce the load from running
975+
the refresh key query itself, not to minimize the number of pre-aggregation
976+
refreshes. The refresh key SQL should be designed to return consistent results
977+
and only change when the underlying data actually changes, ensuring minimal
978+
refreshes even if the query is executed multiple times.
979+
980+
</WarningBox>
981+
967982
#### `incremental`
968983

969984
You can incrementally refresh partitioned rollups by setting
@@ -1241,7 +1256,7 @@ cubes:
12411256
### `scheduled_refresh`
12421257

12431258
To always keep pre-aggregations up-to-date, you can set
1244-
`scheduled_refresh: true`. This option defaults to `true`.
1259+
`scheduled_refresh: true`. This option defaults to `true`.
12451260
In production mode, pre-aggregations with `scheduled_refresh: false` will not be
12461261
built automatically and require external orchestration to trigger their refresh.
12471262
Additionally, any `scheduled_refresh: false` pre-aggregations that were built manually or on-demand will be considered
@@ -1334,8 +1349,8 @@ The SQL queries for the build range (as defined by the `sql` property) are
13341349
executed based on the [`refresh_key`][self-refreshkey] settings of the
13351350
pre-aggregation.
13361351

1337-
In case of very small `update_window` or `FILTER_PARAMS` are used in the [`refresh_key`][self-refreshkey] definition
1338-
and the current timestamp is used as `build_range_end`, there's a possibility
1352+
In case of very small `update_window` or `FILTER_PARAMS` are used in the [`refresh_key`][self-refreshkey] definition
1353+
and the current timestamp is used as `build_range_end`, there's a possibility
13391354
to write [`refresh_key`][self-refreshkey] which won't refresh due to cycle dependency on each other.
13401355
To address such cases, you can use relative date in the future for the `build_range_end`.
13411356
For example, you can add one day to the current timestamp and use it as `build_range_end`.

0 commit comments

Comments
 (0)