clevis with sss using tpm2 and 3 tang servers does not unlock quickly while system is offline

[mknawabi]

I have a system with Ubuntu 22.04 which has /dev/nvme0n1p3 LUKS encrypted, and clevis bound to slot1 with pins for tpm (pcr 7), and 3 tang servers.

I would like the system to unlock quickly when the system does not have network, and I am using t: 1. However, the system hangs for about 4 minutes and 30 seconds before the TPM unlocks. Is there a way to get the TPM to be looked at/prioritized when there is no network available?

GRUB_CMDLINE_LINUX="quiet splash ip:::::eno1:dhcp"

clevis luks bind -f -y -d /dev/sda3 sss '{"t":1,"pins":{"tpm2": {"pcr_ids": "7","pcr_bank":"sha256"},"tang":[{"url":"http://server1.fqdn:7500"},{"url":"http://server2.fqdn:7500"},{"url":"http://server3.fqdn:7500"}]}}'

Please let me know if any further information is needed. I have also tried to inject a curl timeout of 5 into /usr/bin/clevis-decrypt-tang and that also did not work (same 4min 30sec).

Last bit of information, eno1 is not plugged in at the moment.

Thanks much.

[sarroutbi]

AFAIK, if you need SSS for policy reasons (e.g., "TPM2 OR any 1 of 3 Tang servers"), then the current Clevis codebase doesn't support pin prioritization without source code changes.
What you could try is creating a double bind to the device:

• Step 1: Remove existing SSS binding
  sudo clevis luks unbind -d /dev/nvme0n1p3 -s 1

• Step 2: Bind TPM2 to slot 1 (highest priority)
  sudo clevis luks bind -d /dev/nvme0n1p3 -s 1 tpm2 '{"pcr_ids":"7","pcr_bank":"sha256"}'

• Step 3: Bind SSS with Tang servers to slot 2 (network fallback with threshold)

sudo clevis luks bind -d /dev/nvme0n1p3 -s 2 sss '{
    "t": 1,
    "pins": {
      "tang": [
        {"url": "http://server1.fqdn:7500"},
        {"url": "http://server2.fqdn:7500"},
        {"url": "http://server3.fqdn:7500"}
      ]
    }
  }'

• Step 4: Rebuild initramfs
  sudo dracut -f

• Step 5: Verify bindings
  sudo clevis luks list -d /dev/nvme0n1p3

Expected Result:

• Offline: TPM2 (slot 1) unlocks in < 1 second
• TPM2 fails + network: SSS (slot 2) unlocks using any 1 of 3 Tang servers
• TPM2 fails + no network: 4.5-minute timeout, then password prompt

Trade-off:

• Old: If TPM2 is broken but network works, SSS tries both simultaneously
• New: If TPM2 is broken, you wait for TPM2 to fail (~few seconds), then slot 2 SSS Tang kicks in

[mknawabi]

Hi @sarroutbi

Thank you for your suggestion and insight. I have tested this on both Ubuntu 22.04 and Oracle Linux 9 and while the solution seems to work well on OEL9, the Ubuntu+initrd configuration still takes 4m30s to boot. I have tried adding verbose (and removing quiet splash) from the GRUB_CMDLINE_LINUX and regenerating initramfs but do not get any verbose information to go off of. Do you know how I can get more information about the clevis process upon boot on Ubuntu?

Thank you much

[sarroutbi]

Sorry, but I don't have strong knowledge about Ubuntu and initramfs. My suggestion is to try, if possible, to use a newer version (24.04?) to check if that helps

[sergio-correia]

@dannf: could you assist here, please?

[mknawabi]

I have concluded my testing across EL and Ubuntu versions. Using tpm2 in slot 1, and tang (with 3 servers) in slot2, EL8 and EL9 both boot quickly, under 10 seconds. Both Ubuntu 22.04 and 24.04 take longer than 4min30sec to boot, in fact 24.04 took longer.

[dannf]

@dannf: could you assist here, please?

I don't think I'll have time to come up with a fix for it, but I suspect what is going on is that you've told the initramfs that eno1 is expected to come up, and so it is trying hard and waiting a long time to do so. We'd presumably need a context-aware initramfs hook that knew that network was just for this purpose, and to skip if it is already unlocked.

Perhaps an alternative is to see if you can use the dracut support in Ubuntu?
https://discourse.ubuntu.com/t/dracut-will-be-supported-in-ubuntu-25-04/56556

[mknawabi]

@dannf: could you assist here, please?

I don't think I'll have time to come up with a fix for it, but I suspect what is going on is that you've told the initramfs that eno1 is expected to come up, and so it is trying hard and waiting a long time to do so. We'd presumably need a context-aware initramfs hook that knew that network was just for this purpose, and to skip if it is already unlocked.

Perhaps an alternative is to see if you can use the dracut support in Ubuntu? https://discourse.ubuntu.com/t/dracut-will-be-supported-in-ubuntu-25-04/56556

Thanks for your input. I just wanted to add that in my latest testing with the four distributions, I did not include the ip:::::eno1:dhcp portion in the grub configuration. Because we're only supposed to use distributions that are FIPS 140 CMMV validated, our options are quite limited, unfortunately.

[mknawabi]

In further testing, I found that with TPM2 in slot1, and Tang+3 servers in slot2, with a straight out of the box installation config (no specific GRUB options) and with no active network (nothing plugged in), when I comment out the do_configure_networking functional call (line 158) under the clevisloop function, the system boots quickly. So despite the fact that the device is already unlocked, the system waits for network for tang to complete.

So it seems like the change that needs to be made is to the clevis-initramfs package. Is it appropriate to add functionality that checks whether all devices are unlocked before trying to configure networking? I can take a stab at the code, but I don't want to work on something that isn't a "good idea".

Thanks much

[m-ueberall]

So despite the fact that the device is already unlocked, the system waits for network for tang to complete.

If I'm not mistaken, this is exactly the same case as reported in issue #460 (meaning clevis does not kill child processes and exit once the first method succeeded).

[mknawabi]

Hi @m-ueberall,

Can you try commenting out line 158 (do_configure_networking) in /usr/share/initramfs-tools/scripts/local-top/clevis and rerunning update-initramfs -u -k all and see if that removes the delay on your end?

Can you clarify, with the FIDO pin, does it take a long time to eventually unlock or is it stuck?

Thanks

[m-ueberall]

@mknawabi: Using the external FIDO2 pin, there is no delay because said pin does not block resources that would affect the boot process. However, since clevis terminates while the remaining child processes – that is, the other pin instances and their respective children – are still active until they reach their respective timeout, those can delay the boot process. My guess/expectation is that if you properly kill all (clevis) child processes upon exit, you should get rid of all delays regardless of the pins in use.
(If those pins really trigger processes that should survive the termination of clevis, then those processes should explicitly be decoupled anyways; on the other hand, once those pin instances receive a kill signal, they should know what to do in order to free all (blocked) resources only they know about.)