postgres_fdw fallback

[levkk]

This is a big one.

For cross-shard SELECT queries that we don't support yet (some aggregate functions, CTEs / subqueries without the sharding key, OFFSET x, window functions), fallback to using the postgres_fdw extension.

We run an empty Postgres DB as a "sidecar" (subprocess of PgDog, really), and on boot, create all the foreign tables, partitioned by hash/list/range as configured in pgdog.toml. Partitioned tables are supported too, we use two-tier partitioning there, child tables use fdw.

We import custom types, schemas and extensions prior to this, so it should just work(tm).

When a query comes in, pgdog will check that any of the conditions are true, and if so, send the query to the fdw connection pool which will forward it to the shards via fdw, assembling the rows just like it would normally.

This is a superpower. If this works as intended, we will achieve 100% Postgres compatibility in a sharded world.

[codecov]

Codecov Report

❌ Patch coverage is 82.74772% with 491 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pgdog/src/backend/fdw/postgres.rs	79.80%	81 Missing ⚠️
...g/src/frontend/router/parser/query/fdw_fallback.rs	66.66%	78 Missing ⚠️
pgdog/src/backend/schema/postgres_fdw/statement.rs	92.16%	52 Missing ⚠️
pgdog/src/backend/schema/postgres_fdw/schema.rs	77.27%	45 Missing ⚠️
...og/src/backend/schema/postgres_fdw/custom_types.rs	76.78%	39 Missing ⚠️
pgdog/src/backend/fdw/lb.rs	0.00%	32 Missing ⚠️
pgdog/src/backend/databases.rs	35.41%	31 Missing ⚠️
pgdog/src/backend/fdw/launcher.rs	84.57%	31 Missing ⚠️
pgdog/src/backend/pool/cluster.rs	63.26%	18 Missing ⚠️
...gdog/src/backend/schema/postgres_fdw/extensions.rs	73.43%	17 Missing ⚠️
... and 11 more

📢 Thoughts on this report? Let us know!