-
Notifications
You must be signed in to change notification settings - Fork 193
Open
Description
I'm trying to share some config properties via a centralised JavaScript file.
I write some centralised settings into configs.js and spread it into each sqlx config as needed.
Environment: Dataform core (3.0.39), BigQuery
Reproduction (minimal):
- In
includes/configs.js, export a shared config with partitioning:
const cleanTable = {
type: "incremental",
schema: "clean",
bigquery: {
partitionBy: "DATE(event_ts)",
requirePartitionFilter: true,
},
};
module.exports = { cleanTable };- Define two "clean" table models that spread the shared config:
definitions/clean/foo.sqlx
config {
...configs.cleanTable,
name: "foo"
}
select 1 as event_ts;definitions/clean/bar.sqlx
config {
...configs.cleanTable,
name: "bar"
}
select 1 as event_ts;- Run
dataform run --full-refresh.
Expected:
Both clean.foo and clean.bar are created partitioned on event_ts with requirePartitionFilter: true.
Actual:
Only the first table created (foo) is partitioned; the second (bar) is created without partitioning/requirePartitionFilter.
Question:
Is there a "more correct" way to share config setting between sqlx files?
It would be very helpful to have a way of sharing standardised configs across similar tables. Can it be fixed so that it works?
Metadata
Metadata
Assignees
Labels
No labels