Skip to content

Commit c6e8a4c

Browse files
bsanchez-the-roachbsanchez-the-roach
authored
Merge pull request #21819 from cockroachdb/molt-redux-phase-2
Migration doc rework phase 2
2 parents f698b63 + 4a52019 commit c6e8a4c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+1738
-928
lines changed

‎src/current/_includes/molt/migration-create-sql-user.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ ALTER DEFAULT PRIVILEGES IN SCHEMA public
4545
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO crdb_user;
4646
~~~
4747

48-
Depending on the MOLT Fetch [data load mode](#data-load-mode) you will use, grant the necessary privileges to run either [`IMPORT INTO`](#import-into-privileges) or [`COPY FROM`](#copy-from-privileges) on the target tables:
48+
Depending on the MOLT Fetch [data load mode]({% link molt/molt-fetch.md %}#import-into-vs-copy-from) you will use, grant the necessary privileges to run either [`IMPORT INTO`](#import-into-privileges) or [`COPY FROM`](#copy-from-privileges) on the target tables:
4949

5050
#### `IMPORT INTO` privileges
5151

‎src/current/_includes/molt/migration-schema-design-practices.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,12 +31,12 @@ Convert the source table definitions into CockroachDB-compatible equivalents. Co
3131
~~~
3232
</section>
3333

34-
- MOLT Fetch can automatically define matching CockroachDB tables using the {% if page.name != "migration-strategy.md" %}[`drop-on-target-and-recreate`](#table-handling-mode){% else %}[`drop-on-target-and-recreate`]({% link molt/molt-fetch.md %}#target-table-handling){% endif %} option.
34+
- MOLT Fetch can automatically define matching CockroachDB tables using the {% if page.name != "migration-strategy.md" %}[`drop-on-target-and-recreate`](#table-handling-mode){% else %}[`drop-on-target-and-recreate`]({% link molt/molt-fetch.md %}#handle-target-tables){% endif %} option.
3535

3636
- If you define the target tables manually, review how MOLT Fetch handles [type mismatches]({% link molt/molt-fetch.md %}#mismatch-handling). You can use the {% if page.name != "migration-strategy.md" %}[MOLT Schema Conversion Tool](#schema-conversion-tool){% else %}[MOLT Schema Conversion Tool]({% link cockroachcloud/migrations-page.md %}){% endif %} to create matching table definitions.
3737

3838
<section class="filter-content" markdown="1" data-scope="oracle">
39-
- By default, table and column names are case-insensitive in MOLT Fetch. If using the [`--case-sensitive`]({% link molt/molt-fetch.md %}#global-flags) flag, schema, table, and column names must match Oracle's default uppercase identifiers. Use quoted names on the target to preserve case. For example, the following CockroachDB SQL statement will error:
39+
- By default, table and column names are case-insensitive in MOLT Fetch. If using the [`--case-sensitive`]({% link molt/molt-fetch-commands-and-flags.md %}#global-flags) flag, schema, table, and column names must match Oracle's default uppercase identifiers. Use quoted names on the target to preserve case. For example, the following CockroachDB SQL statement will error:
4040

4141
~~~ sql
4242
CREATE TABLE co.stores (... store_id ...);
@@ -57,6 +57,6 @@ Convert the source table definitions into CockroachDB-compatible equivalents. Co
5757
Avoid using sequential keys. To learn more about the performance issues that can result from their use, refer to the [guidance on indexing with sequential keys]({% link {{site.current_cloud_version}}/sql-faqs.md %}#how-do-i-generate-unique-slowly-increasing-sequential-numbers-in-cockroachdb). If a sequential key is necessary in your CockroachDB table, you must create it manually, after using [MOLT Fetch]({% link molt/molt-fetch.md %}) to load and replicate the data.
5858
{{site.data.alerts.end}}
5959

60-
- Review [Transformations]({% link molt/molt-fetch.md %}#transformations) to understand how computed columns and partitioned tables can be mapped to the target, and how target tables can be renamed.
60+
- Review [Transformations]({% link molt/molt-fetch.md %}#define-transformations) to understand how computed columns and partitioned tables can be mapped to the target, and how target tables can be renamed.
6161

6262
- By default on CockroachDB, `INT` is an alias for `INT8`, which creates 64-bit signed integers. PostgreSQL and MySQL default to 32-bit integers. Depending on your source database or application requirements, you may need to change the integer size to `4`. For more information, refer to [Considerations for 64-bit signed integers]({% link {{ site.current_cloud_version }}/int.md %}#considerations-for-64-bit-signed-integers).

‎src/current/_includes/molt/molt-drop-constraints-indexes.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
To optimize data load performance, drop all non-`PRIMARY KEY` [constraints]({% link {{ site.current_cloud_version }}/alter-table.md %}#drop-constraint) and [indexes]({% link {{site.current_cloud_version}}/drop-index.md %}) on the target CockroachDB database before migrating:
2-
{% if page.name == "molt-fetch.md" %}
2+
{% if page.name == "molt-fetch-best-practices.md" %}
33
- [`FOREIGN KEY`]({% link {{ site.current_cloud_version }}/foreign-key.md %})
44
- [`UNIQUE`]({% link {{ site.current_cloud_version }}/unique.md %})
55
- [Secondary indexes]({% link {{ site.current_cloud_version }}/schema-design-indexes.md %})
66
- [`CHECK`]({% link {{ site.current_cloud_version }}/check.md %})
77
- [`DEFAULT`]({% link {{ site.current_cloud_version }}/default-value.md %})
8-
- [`NOT NULL`]({% link {{ site.current_cloud_version }}/not-null.md %}) (you do not need to drop this constraint when using `drop-on-target-and-recreate` for [table handling](#target-table-handling))
8+
- [`NOT NULL`]({% link {{ site.current_cloud_version }}/not-null.md %}) (you do not need to drop this constraint when using `drop-on-target-and-recreate` for [table handling]({% link molt/molt-fetch.md %}#handle-target-tables))
99

1010
{{site.data.alerts.callout_danger}}
1111
Do **not** drop [`PRIMARY KEY`]({% link {{ site.current_cloud_version }}/primary-key.md %}) constraints.

‎src/current/_includes/molt/molt-limitations.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
- Oracle advises against `LONG RAW` columns and [recommends converting them to `BLOB`](https://www.orafaq.com/wiki/LONG_RAW#History). `LONG RAW` can only store binary values up to 2GB, and only one `LONG RAW` column per table is supported.
1313
</section>
1414

15-
- Only tables with [primary key]({% link {{ site.current_cloud_version }}/primary-key.md %}) types of [`INT`]({% link {{ site.current_cloud_version }}/int.md %}), [`FLOAT`]({% link {{ site.current_cloud_version }}/float.md %}), or [`UUID`]({% link {{ site.current_cloud_version }}/uuid.md %}) can be sharded with [`--export-concurrency`]({% link molt/molt-fetch.md %}#best-practices).
15+
- Only tables with [primary key]({% link {{ site.current_cloud_version }}/primary-key.md %}) types of [`INT`]({% link {{ site.current_cloud_version }}/int.md %}), [`FLOAT`]({% link {{ site.current_cloud_version }}/float.md %}), or [`UUID`]({% link {{ site.current_cloud_version }}/uuid.md %}) can be sharded with [`--export-concurrency`]({% link molt/molt-fetch-best-practices.md %}#configure-the-source-database-and-connection).
1616

1717
{% if page.name != "migrate-bulk-load.md" %}
1818
#### Replicator limitations
@@ -37,5 +37,5 @@
3737

3838
- Running DDL on the source or target while replication is in progress can cause replication failures.
3939
- `TRUNCATE` operations on the source are not captured. Only `INSERT`, `UPDATE`, `UPSERT`, and `DELETE` events are replicated.
40-
- Changes to virtual columns are not replicated automatically. To migrate these columns, you must define them explicitly with [transformation rules]({% link molt/molt-fetch.md %}#transformations).
40+
- Changes to virtual columns are not replicated automatically. To migrate these columns, you must define them explicitly with [transformation rules]({% link molt/molt-fetch.md %}#define-transformations).
4141
{% endif %}

‎src/current/_includes/molt/molt-setup.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
- Create a CockroachDB [{{ site.data.products.cloud }}]({% link cockroachcloud/create-your-cluster.md %}) or [{{ site.data.products.core }}]({% link {{ site.current_cloud_version }}/install-cockroachdb-mac.md %}) cluster.
1111
- Install the [MOLT (Migrate Off Legacy Technology)]({% link releases/molt.md %}#installation) tools.
12-
- Review the [Fetch]({% link molt/molt-fetch.md %}#best-practices) and {% if page.name != "migrate-bulk-load.md" %}[Replicator]({% link molt/molt-replicator.md %}#best-practices){% endif %} best practices.
12+
- Review the [Fetch]({% link molt/molt-fetch-best-practices.md %}) and {% if page.name != "migrate-bulk-load.md" %}[Replicator]({% link molt/molt-replicator-best-practices.md %}){% endif %} best practices.
1313
- Review [Migration Strategy]({% link molt/migration-strategy.md %}).
1414

1515
<section class="filter-content" markdown="1" data-scope="oracle">

‎src/current/_includes/releases/v24.2/feature-highlights-migrations.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
MOLT Fetch transformation rules
3232
</p>
3333
<p class="feature-description">
34-
Column exclusion, computed columns, and partitioned tables are now supported in table migrations with MOLT Fetch. They are supported via a new <a href="https://www.cockroachlabs.com/docs/molt/molt-fetch.html#transformations">transformations framework</a> that allows the user to specify a JSON file with instructions on how MOLT Fetch should treat certain schemas, tables, or underlying columns.
34+
Column exclusion, computed columns, and partitioned tables are now supported in table migrations with MOLT Fetch. They are supported via a new <a href="https://www.cockroachlabs.com/docs/molt/molt-fetch.html#define-transformations">transformations framework</a> that allows the user to specify a JSON file with instructions on how MOLT Fetch should treat certain schemas, tables, or underlying columns.
3535
</p>
3636
</td>
3737
<td>All<sup>&starf;&starf;</sup>

‎src/current/_includes/v23.1/sidebar-data/migrate.json‎

Lines changed: 76 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,14 +54,60 @@
5454
"items": [
5555
{
5656
"title": "Schema Conversion Tool",
57-
"urls": [
58-
"/cockroachcloud/migrations-page.html"
57+
"items": [
58+
{
59+
"title": "Guide",
60+
"urls": [
61+
"/cockroachcloud/migrations-page.html"
62+
]
63+
},
64+
{
65+
"title": "Type Mapping",
66+
"urls": [
67+
"/molt/molt-type-mapping.html"
68+
]
69+
}
5970
]
6071
},
6172
{
6273
"title": "Fetch",
63-
"urls": [
64-
"/molt/molt-fetch.html"
74+
"items": [
75+
{
76+
"title": "Guide",
77+
"urls": [
78+
"/molt/molt-fetch.html"
79+
]
80+
},
81+
{
82+
"title": "Installation",
83+
"urls": [
84+
"/molt/molt-fetch-installation.html"
85+
]
86+
},
87+
{
88+
"title": "Commands and Flags",
89+
"urls": [
90+
"/molt/molt-fetch-commands-and-flags.html"
91+
]
92+
},
93+
{
94+
"title": "Metrics",
95+
"urls": [
96+
"/molt/molt-fetch-monitoring.html"
97+
]
98+
},
99+
{
100+
"title": "Best Practices",
101+
"urls": [
102+
"/molt/molt-fetch-best-practices.html"
103+
]
104+
},
105+
{
106+
"title": "Troubleshooting",
107+
"urls": [
108+
"/molt/molt-fetch-troubleshooting.html"
109+
]
110+
}
65111
]
66112
},
67113
{
@@ -74,7 +120,13 @@
74120
]
75121
},
76122
{
77-
"title": "Flags",
123+
"title": "Installation",
124+
"urls": [
125+
"/molt/molt-replicator-installation.html"
126+
]
127+
},
128+
{
129+
"title": "Commands and Flags",
78130
"urls": [
79131
"/molt/replicator-flags.html"
80132
]
@@ -84,6 +136,18 @@
84136
"urls": [
85137
"/molt/replicator-metrics.html"
86138
]
139+
},
140+
{
141+
"title": "Best Practices",
142+
"urls": [
143+
"/molt/molt-replicator-best-practices.html"
144+
]
145+
},
146+
{
147+
"title": "Troubleshooting",
148+
"urls": [
149+
"/molt/molt-replicator-troubleshooting.html"
150+
]
87151
}
88152
]
89153
},
@@ -99,33 +163,33 @@
99163
"title": "Common Migration Approaches",
100164
"items": [
101165
{
102-
"title": "Classic Bulk Load",
166+
"title": "Classic Bulk Load Migration",
103167
"urls": [
104-
"/molt/migrate-bulk-load.html"
168+
"/molt/migration-approach-classic-bulk-load.html"
105169
]
106170
},
107171
{
108-
"title": "Phased Bulk Load",
172+
"title": "Phased Bulk Load Migration",
109173
"urls": [
110-
"/molt/migrate-bulk-load.html"
174+
"/molt/migration-approach-phased-bulk-load.html"
111175
]
112176
},
113177
{
114178
"title": "Delta Migration",
115179
"urls": [
116-
"/molt/migrate-load-replicate.html"
180+
"/molt/migration-approach-delta.html"
117181
]
118182
},
119183
{
120184
"title": "Streaming Migration",
121185
"urls": [
122-
"/molt/migrate-failback.html"
186+
"/molt/migration-approach-streaming.html"
123187
]
124188
},
125189
{
126190
"title": "Active-Active Migration",
127191
"urls": [
128-
"/molt/migrate-failback.html"
192+
"/molt/migration-approach-active-active.html"
129193
]
130194
}
131195
]

‎src/current/_includes/v23.2/sidebar-data/migrate.json‎

Lines changed: 76 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,14 +54,60 @@
5454
"items": [
5555
{
5656
"title": "Schema Conversion Tool",
57-
"urls": [
58-
"/cockroachcloud/migrations-page.html"
57+
"items": [
58+
{
59+
"title": "Guide",
60+
"urls": [
61+
"/cockroachcloud/migrations-page.html"
62+
]
63+
},
64+
{
65+
"title": "Type Mapping",
66+
"urls": [
67+
"/molt/molt-type-mapping.html"
68+
]
69+
}
5970
]
6071
},
6172
{
6273
"title": "Fetch",
63-
"urls": [
64-
"/molt/molt-fetch.html"
74+
"items": [
75+
{
76+
"title": "Guide",
77+
"urls": [
78+
"/molt/molt-fetch.html"
79+
]
80+
},
81+
{
82+
"title": "Installation",
83+
"urls": [
84+
"/molt/molt-fetch-installation.html"
85+
]
86+
},
87+
{
88+
"title": "Commands and Flags",
89+
"urls": [
90+
"/molt/molt-fetch-commands-and-flags.html"
91+
]
92+
},
93+
{
94+
"title": "Metrics",
95+
"urls": [
96+
"/molt/molt-fetch-monitoring.html"
97+
]
98+
},
99+
{
100+
"title": "Best Practices",
101+
"urls": [
102+
"/molt/molt-fetch-best-practices.html"
103+
]
104+
},
105+
{
106+
"title": "Troubleshooting",
107+
"urls": [
108+
"/molt/molt-fetch-troubleshooting.html"
109+
]
110+
}
65111
]
66112
},
67113
{
@@ -74,7 +120,13 @@
74120
]
75121
},
76122
{
77-
"title": "Flags",
123+
"title": "Installation",
124+
"urls": [
125+
"/molt/molt-replicator-installation.html"
126+
]
127+
},
128+
{
129+
"title": "Commands and Flags",
78130
"urls": [
79131
"/molt/replicator-flags.html"
80132
]
@@ -84,6 +136,18 @@
84136
"urls": [
85137
"/molt/replicator-metrics.html"
86138
]
139+
},
140+
{
141+
"title": "Best Practices",
142+
"urls": [
143+
"/molt/molt-replicator-best-practices.html"
144+
]
145+
},
146+
{
147+
"title": "Troubleshooting",
148+
"urls": [
149+
"/molt/molt-replicator-troubleshooting.html"
150+
]
87151
}
88152
]
89153
},
@@ -99,33 +163,33 @@
99163
"title": "Common Migration Approaches",
100164
"items": [
101165
{
102-
"title": "Classic Bulk Load",
166+
"title": "Classic Bulk Load Migration",
103167
"urls": [
104-
"/molt/migrate-bulk-load.html"
168+
"/molt/migration-approach-classic-bulk-load.html"
105169
]
106170
},
107171
{
108-
"title": "Phased Bulk Load",
172+
"title": "Phased Bulk Load Migration",
109173
"urls": [
110-
"/molt/migrate-bulk-load.html"
174+
"/molt/migration-approach-phased-bulk-load.html"
111175
]
112176
},
113177
{
114178
"title": "Delta Migration",
115179
"urls": [
116-
"/molt/migrate-load-replicate.html"
180+
"/molt/migration-approach-delta.html"
117181
]
118182
},
119183
{
120184
"title": "Streaming Migration",
121185
"urls": [
122-
"/molt/migrate-failback.html"
186+
"/molt/migration-approach-streaming.html"
123187
]
124188
},
125189
{
126190
"title": "Active-Active Migration",
127191
"urls": [
128-
"/molt/migrate-failback.html"
192+
"/molt/migration-approach-active-active.html"
129193
]
130194
}
131195
]

0 commit comments

Comments
 (0)