add bsetup step to README, and some other suggestions #8518
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
acaird
changed the title
aarongable
requested changes
Dec 8, 2025
Contributor
aarongable
left a comment
aarongable
left a comment
There was a problem hiding this comment.
Thanks! I'd propose a slightly different edit: let's just move the docker compose up instruction to the end, after all the instructions on how to run the tests. In general, running the tests is much more common than simply starting the container. And if we move the docker compose up instruction to much later in the doc, then we don't have to include all these caveats about running docker compose down before running the tests.
I'm also not a fan of including example error messages -- because our readme is so simple, they're presented in the exact same format as instructions on what to run, and that can get confusing.
README.md
Outdated
| Before starting Boulder for the first time, run: | ||
|
|
||
| ```shell | ||
| docker compose up bsetup |
Contributor
There was a problem hiding this comment.
This should be docker compose run bsetup, no need to leave this container running.
Contributor
Author
There was a problem hiding this comment.
I made the changes you suggested; please take another look.
aarongable
force-pushed
the
acaird/update-readme-for-bsetup
branch
from
4cb6497 to
6dd6dc9
Compare
aarongable
reviewed
Dec 10, 2025
Contributor
aarongable
left a comment
aarongable
left a comment
There was a problem hiding this comment.
Looks like that merge got a bit messed up and was recapitulating diffs from already-merged changes, so I fixed that for you.
README.md
Outdated
Comment on lines
106
to
116
| Before starting Boulder for the first time, run: | ||
|
|
||
| ```shell | ||
| docker compose up | ||
| docker compose run bsetup | ||
| ``` | ||
| this will write the necessary certificates into | ||
| `test/certs/[.softhsm-tokens,ipki,webpki]`; you only need to run this | ||
| once to create the certificates. If you need to remove all of the | ||
| certificates and start over, you can remove the directories | ||
| `./test/certs/.softhsm-tokens`, `./test/certs/ipki`, and | ||
| `./test/certs/webpki` and re-run `docker compose run bsetup`. |
Contributor
There was a problem hiding this comment.
This should move down to around line 221, since it's only necessary if you want to run docker compose up. All of the test scripts run bsetup automatically, so this step doesn't apply if all you want to do is run the tests.
acaird
force-pushed
the
acaird/update-readme-for-bsetup
branch
from
e44ce11 to
7116e88
Compare
The new "created" column is always populated by the current time, indicating when the row was created. We do this directly inside the database, rather than in the Go data model, because Boulder doesn't actually care when these rows were created. Instead, we care about the creation timestamp for the purpose of creating and dropping partitions to manage our total data size. Previously we've been partitioning these tables based on their `expires` column. But that column is no longer a good proxy for when the row was created, because not all certificates/orders/authzs have the same lifetime anymore. Example after an integration test run: ```mysql MariaDB [boulder_sa_integration]> select id,created,expires from orderFqdnSets limit 5; +----+---------------------+---------------------+ | id | created | expires | +----+---------------------+---------------------+ | 1 | 2025-12-04 00:49:10 | 2025-12-11 00:49:10 | | 3 | 2025-12-04 00:49:12 | 2025-12-11 00:49:12 | | 6 | 2025-12-04 00:49:17 | 2025-12-11 00:49:17 | | 9 | 2025-12-04 00:49:23 | 2025-12-11 00:49:23 | | 10 | 2025-12-04 00:49:24 | 2025-12-11 00:49:24 | +----+---------------------+---------------------+ 5 rows in set (0.000 sec) ``` IN-12028 tracks the corresponding SRE-side changes
…8514) In 978877a35 ("Store authzIDs directly in order table"), the modelToOrder function was updated to decode the new Authzs protobuf column, but the decoded values were never assigned to the order's V2Authorizations field. This caused the StoreAuthzsInOrders feature to not work correctly for reads. The bug meant that even with StoreAuthzsInOrders enabled: 1. Reads always fell back to querying the orderToAuthz2 table 2. The Authzs blob was parsed but the result was discarded 3. Eventually dropping orderToAuthz2 would break all order reads The fix assigns the decoded authz IDs to V2Authorizations, allowing orders with the new column populated to skip the fallback query. Also adds TestModelToOrderAuthzs to verify the fix and prevent regression.
BR 7.1.3.2.2 states that only ECDSA curves P-256, P-384, and P-512 are allowed. This should prevent a silly footgun.
…8510) For our purpose (of testing non-production logs), these are equivalent.
The original plan for getting the Vitess infrastructure running was to use [vttestserver](https://vitess.io/docs/22.0/reference/programs/vttestserver) as a starting point to reach a minimum viable setup. However, vttestserver didn’t work out because some of its defaults conflicted with how we clean up rows and the level of resources (threads) we need. Fortunately, vttestserver is just a wrapper around [vtcombo](https://vitess.io/docs/21.0/reference/programs/vtcombo) that generates a [vttest protobuf](https://github.com/vitessio/vitess/blob/v22.0.1/proto/vttest.proto) describing the configuration for an in-memory topology server started by vtcombo, encoded in JSON. By modifying vttestserver’s [run.sh](https://github.com/vitessio/vitess/blob/v22.0.1/docker/vttestserver/run.sh), we're able to interact with vtcombo directly, passing the JSON configuration along with other vttestserver defaults reverse-engineered from run.sh and [vtprocess.go](https://github.com/vitessio/vitess/blob/v22.0.1/go/vt/vttest/vtprocess.go). Vitess doesn’t provide a `vtcombo` image, we must build our own. Build and upload a [boulder-vtcomboserver image](https://hub.docker.com/repository/docker/letsencrypt/boulder-vtcomboserver) on top of Docker's official MySQL 8.4 image, which provides native arm64 support. The accompanying tag-and-upload shell script defaults to amd64 for CI. As an aside, Vitess’s official Dockerfiles are only published for amd64, and modifying them to build for arm64 would prove difficult because Oracle doesn’t publish MySQL arm64 binaries in its [Debian apt repository](https://repo.mysql.com/apt/debian/pool/mysql-8.0/m/mysql-community). With boulder-vtcomboserver up and running I was able to find/validate the following issues and provide workarounds: - **Problem:** db-migrate, the tool we use to apply database migrations, must be configured to talk to MariaDB and to MySQL through Vitess (vtgate + vttablet). **Solution:** Create two new dbconfig YAML files (mariadb and vitess) and use `test/entrypoint.sh` to set the appropriate file for `sql-migrate` (`test/create_db.sh`) to use. Also, symlink each of these two new files from db to db-next just like the old dbconfig.yml file. - **Problem:** Vitess does not allow database `CREATE` statements and any DDL containing them will be rejected by vtgate. **Solution:** These databases are already created by vtcombo since they’re defined as KEYSPACES. Skip database creation in `test/create_db.sh`. - **Problem:** Vitess does not allow user creation or grants (`CREATE USER`, `GRANT`), and any DDL containing these commands will be blocked by vtgate. **Solution:** Skip user creation and grant steps in `test/create_db.sh`. Set `%` for `--vschema_ddl_authorized_users` as vttestserver does, and revisit this later for a more complete approach. - **Problem:** vttablet default for maximum number of rows returned from a (non-streaming) query (10,000) is too low for Boulder’s needs, causing queries to fail due to vttablet rejecting them. **Solution:** Increase `--queryserver-config-max-result-size` to 1,000,000 and `--queryserver-config-warn-result-size` to 1,000,000. - **Problem:** vttablet default for connection pool size (16) and maximum number of concurrent transactions (20) are too low for Boulder’s needs, causing queries to fail due to vttablet being overloaded. **Solution:** Increase `--queryserver-config-pool-size` to 64 and `--queryserver-config-transaction-cap` to 80. - **Problem:** Vitess does not allow `TRIGGER` statements and any DDL containing them will be rejected by vtgate. Without TRIGGER statements TestIssuanceCertStorageFailed, an integration test, will fail. **Soluton:** Run these TRIGGER statements in an entrypoint scripttest/vtcomboserver/install_trigger.sh, bypassing vtgate entirely. Depends on #8479 Depends on #8489 Depends on #8490 Depends on #8494 Fixes #7736
acaird
force-pushed
the
acaird/update-readme-for-bsetup
branch
from
394674a to
1ce972b
Compare
) Bumps the aws group with 3 updates in the / directory: [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2). Updates `github.com/aws/aws-sdk-go-v2` from 1.39.2 to 1.39.3 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/b9b0c6553b80f99603b4f8356b88f5baf1328deb"><code>b9b0c65</code></a> Release 2025-10-16</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/e2bc8a0ec6f430876fc7de4432ea9cc89c9568f8"><code>e2bc8a0</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/8691ee380a96c49351e4b5ab8a70bc5d4d100724"><code>8691ee3</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/51e8a3fe032fc566d31b389f492ab58475a98398"><code>51e8a3f</code></a> bump to go1.23 (<a href="https://redirect.github.com/aws/aws-sdk-go-v2/issues/3211">#3211</a>)</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/ad2d36cba7c5772b4e8e4caf96939dc41b95c65c"><code>ad2d36c</code></a> Release 2025-10-15</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/19a35d639f969ee328553e632e8cf8b83d324106"><code>19a35d6</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/35cb02fd50fb125601b9c3b33feb72f3a2bcaa56"><code>35cb02f</code></a> Update endpoints model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/f673a1b0a80e666c0128ec606ff053dace9771f1"><code>f673a1b</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/48421fd812d8592a4eb2b32d11ae07e228969012"><code>48421fd</code></a> Release 2025-10-14</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/fedcba778c21b451a91b4e4bcdd5d6c1554c6a5a"><code>fedcba7</code></a> Regenerated Clients</li> <li>Additional commits viewable in <a href="https://github.com/aws/aws-sdk-go-v2/compare/v1.39.2...v1.39.3">compare view</a></li> </ul> </details> <br /> Updates `github.com/aws/aws-sdk-go-v2/config` from 1.31.11 to 1.31.13 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/b9b0c6553b80f99603b4f8356b88f5baf1328deb"><code>b9b0c65</code></a> Release 2025-10-16</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/e2bc8a0ec6f430876fc7de4432ea9cc89c9568f8"><code>e2bc8a0</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/8691ee380a96c49351e4b5ab8a70bc5d4d100724"><code>8691ee3</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/51e8a3fe032fc566d31b389f492ab58475a98398"><code>51e8a3f</code></a> bump to go1.23 (<a href="https://redirect.github.com/aws/aws-sdk-go-v2/issues/3211">#3211</a>)</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/ad2d36cba7c5772b4e8e4caf96939dc41b95c65c"><code>ad2d36c</code></a> Release 2025-10-15</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/19a35d639f969ee328553e632e8cf8b83d324106"><code>19a35d6</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/35cb02fd50fb125601b9c3b33feb72f3a2bcaa56"><code>35cb02f</code></a> Update endpoints model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/f673a1b0a80e666c0128ec606ff053dace9771f1"><code>f673a1b</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/48421fd812d8592a4eb2b32d11ae07e228969012"><code>48421fd</code></a> Release 2025-10-14</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/fedcba778c21b451a91b4e4bcdd5d6c1554c6a5a"><code>fedcba7</code></a> Regenerated Clients</li> <li>Additional commits viewable in <a href="https://github.com/aws/aws-sdk-go-v2/compare/config/v1.31.11...config/v1.31.13">compare view</a></li> </ul> </details> <br /> Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.88.3 to 1.88.5 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/b9b0c6553b80f99603b4f8356b88f5baf1328deb"><code>b9b0c65</code></a> Release 2025-10-16</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/e2bc8a0ec6f430876fc7de4432ea9cc89c9568f8"><code>e2bc8a0</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/8691ee380a96c49351e4b5ab8a70bc5d4d100724"><code>8691ee3</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/51e8a3fe032fc566d31b389f492ab58475a98398"><code>51e8a3f</code></a> bump to go1.23 (<a href="https://redirect.github.com/aws/aws-sdk-go-v2/issues/3211">#3211</a>)</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/ad2d36cba7c5772b4e8e4caf96939dc41b95c65c"><code>ad2d36c</code></a> Release 2025-10-15</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/19a35d639f969ee328553e632e8cf8b83d324106"><code>19a35d6</code></a> Regenerated Clients</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/35cb02fd50fb125601b9c3b33feb72f3a2bcaa56"><code>35cb02f</code></a> Update endpoints model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/f673a1b0a80e666c0128ec606ff053dace9771f1"><code>f673a1b</code></a> Update API model</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/48421fd812d8592a4eb2b32d11ae07e228969012"><code>48421fd</code></a> Release 2025-10-14</li> <li><a href="https://github.com/aws/aws-sdk-go-v2/commit/fedcba778c21b451a91b4e4bcdd5d6c1554c6a5a"><code>fedcba7</code></a> Regenerated Clients</li> <li>Additional commits viewable in <a href="https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.88.3...service/s3/v1.88.5">compare view</a></li> </ul> </details> <br /> Updates `github.com/aws/smithy-go` from 1.23.0 to 1.23.1 <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/aws/smithy-go/commit/695755c8eb501de6899f71cd19e8249daff8f8b9"><code>695755c</code></a> Release 2025-10-15</li> <li><a href="https://github.com/aws/smithy-go/commit/4355cab7372c2b952d022bb4403b5e2bc71d0bc5"><code>4355cab</code></a> changelog</li> <li><a href="https://github.com/aws/smithy-go/commit/40bb509431e08e9d10d2baaa7d27c171421cc8d8"><code>40bb509</code></a> bump to go1.23 (<a href="https://redirect.github.com/aws/smithy-go/issues/606">#606</a>)</li> <li><a href="https://github.com/aws/smithy-go/commit/0ccf069f0a011a44ee83477748e637a0e3898d7e"><code>0ccf069</code></a> Update Smithy to 1.62.0 (<a href="https://redirect.github.com/aws/smithy-go/issues/605">#605</a>)</li> <li><a href="https://github.com/aws/smithy-go/commit/04e063a48301226f81d645e0b899f32a6455ef63"><code>04e063a</code></a> Release 2025-09-18</li> <li><a href="https://github.com/aws/smithy-go/commit/fd9193a890d1d7234d18fff1ff102bd1f343409d"><code>fd9193a</code></a> changelog</li> <li><a href="https://github.com/aws/smithy-go/commit/1ec19f0d413b14ae9f8f6f0f5abc72e09d5666a4"><code>1ec19f0</code></a> Added support for SIG4/SIGV4A querystring authentication. (<a href="https://redirect.github.com/aws/smithy-go/issues/595">#595</a>)</li> <li><a href="https://github.com/aws/smithy-go/commit/62e8fe91fa931af8d4f2a96f367d88fbf52476df"><code>62e8fe9</code></a> Make <code>make</code> commands multi-module aware (<a href="https://redirect.github.com/aws/smithy-go/issues/600">#600</a>)</li> <li><a href="https://github.com/aws/smithy-go/commit/5ef2d9617cb8ad8abc352e3fd89acf9c9427e5fe"><code>5ef2d96</code></a> Allow unused required parameters in endpoint rule set (<a href="https://redirect.github.com/aws/smithy-go/issues/601">#601</a>)</li> <li>See full diff in <a href="https://github.com/aws/smithy-go/compare/v1.23.0...v1.23.1">compare view</a></li> </ul> </details> <br /> <details> <summary>Most Recent Ignore Conditions Applied to This Pull Request</summary> | Dependency Name | Ignore Conditions | | --- | --- | | github.com/aws/aws-sdk-go-v2/service/s3 | [< 1.28, > 1.27.1] | | github.com/aws/aws-sdk-go-v2/config | [< 1.18, > 1.17.1] | | github.com/aws/aws-sdk-go-v2/service/s3 | [< 1.31, > 1.30.5] | </details> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
While constant-time comparison is important in cryptographic algorithms, that's not what we're doing here. The validation random token is not intended to be secret in the same way as (say) a private key is: it's just meant to be random enough that it's unlikely to exist in DNS or on a webserver by chance. Possession of the random token does not give an attacker any advantages; they still need to control the domain itself, at which point they could get a random token of their own. Using subtle.ConstantTimeCompare is overkill and sets a bad example for places that truly do need to use it.
Enable local builds of the vtcomboserver image from test/vtcomboserver/ while preserving the existing pull behavior via BOULDER_VTCOMBOSERVER_TAG.
Update ra.NewOrder to allow reuse of authorizations with a dns-account-01 challenge for wildcard identifiers. Add two new tests showing that the RA accepts or rejects reuse of such an authorization, depending on the state of the DNSAccount01Enabled feature flag. Fixes #8505
- add `docker compose run bsetup` step to README, with some explanation of when to run it and how to clean up what it does - add some advice on running the tests Fixes #8517
acaird
force-pushed
the
acaird/update-readme-for-bsetup
branch
from
1ce972b to
e45e072
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add
docker compose up bsetupstep to README, with some explanation of when to run it and how to clean up what it doesadd some advice on running the tests
add some error messages to the README so if people search for them, hopefully they will find them here and then actually read the README
Fixes #8517