is haveged still useful/relevant?

[klausenbusk]

Hi

Sorry for the harsh question, but with the jitter entropy added in v5.4 (merge commit, commit, LKML) and removal of the /dev/random blocking pool in v5.6 (commit, LWN), is haveged still useful/relevant? If yes, under what circumstances?

- Kristian

[jirka-h]

Hi Kristian,

thanks a lot for pointing me to the recent kernel development. 

After reading the LKML/LWN articles, I completely agree that the haveged service is now obsolete (starting from kernel 5.6). I have verified it experimentally on Fedora 32, running kernel 5.10

 $time timeout 30 pv /dev/random > /dev/null
1.18GiB 0:00:29 [42.2MiB/s]
real    0m30.012s
user    0m0.022s
sys     0m29.934s

1. There is no difference in throughput with and without haveged service running
2. haveged service is not triggered at all as verified with strace. No entropy is sent to the kernel.

I'm happy that these changes made it into the mainline kernel. It's nice to see that the main idea behind HAVEGED has sustained time test! (It was published already in 2003 here: https://www.irisa.fr/caps/projects/hipsor/publications/havege-tomacs.pdf)

I'm also glad that the HAVEGE algorithm is being further explored and examined - see the "CPU Jitter Random Number Generator" page at https://www.chronox.de/jent.html

I will keep maintaining HAVEGED - most Linux installations are still running on the older kernel versions. HAVEGED can also be used as the userspace RNG to generate random numbers. See man -S8 haveged for examples or try running haveged -n 0 | pv > /dev/null

Last but not least, HAVEGED can be used as the RNG library.

Thanks a lot
Jirka

Further references:
https://lore.kernel.org/lkml/alpine.DEB.2.21.1909290010500.2636@nanos.tec.linutronix.de/T/
https://lwn.net/Articles/808575/
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.6-Random-Rework
https://en.wikipedia.org/wiki//dev/random
https://www.irisa.fr/caps/projects/hipsor/publications/havege-tomacs.pdf
https://www.chronox.de/jent.html
https://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.pdf
https://github.com/sandy-harris/maxwell

[eworm-de]

Relevant info has been added in 297bdf1, also cef1d42 adds a condition on kernel version in service file and makes it a no-op for linux >= 5.6.

[frostschutz]

So now with haveged deemed "obsolete" done and getting dropped all over the place because it doesn't do anything, how does your average joe linux hobbyist admin deal with crng init taking ages? That's still happening in recent kernels... and was the prime reason I installed haveged pretty much everywhere, only it's a no-op now.

Bare metal (archlinux kernel 5.15):

dmesg | grep random
[    0.287693] random: get_random_u64 called from __kmem_cache_create+0x2a/0x540 with crng_init=0
[    6.158634] random: fast init done
[  100.703821] random: crng init done

In a KVM virtual machine (archlinux kernel 5.10):

[    0.063991] random: get_random_u64 called from __kmem_cache_create+0x2a/0x4c0 with crng_init=0
[    1.558234] random: fast init done
[    3.958456] random: cryptsetup: uninitialized urandom read (4 bytes read)
[   36.354001] random: cryptsetup: uninitialized urandom read (32 bytes read)
[   41.265375] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   41.265477] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   41.265481] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   50.726303] random: crng init done
[   50.726321] random: 5 urandom warning(s) missed due to ratelimiting

Same virtual machine with manually tickling the random device early in initramfs:

dmesg | grep random
[    0.046587] random: get_random_u64 called from __kmem_cache_create+0x2a/0x4c0 with crng_init=0
[    1.377856] random: fast init done
[    2.696946] random: crng init done

So massaging the random device early still has an influence, and I'd love to continue using haveged for the job, but for that to work it would actually have to do something, like unconditionally feed some randomness on startup, either as a once-off or periodically... as far as random sources go, it's the more the merrier, isn't it?

[jirka-h]

Hi Andreas,

I have updated haveged to feed entropy to the kernel on start and then every 60 seconds. Please give it a try and let me know if it works for you. See commits b0d1b0e and c35c6f4

I have tested it on x86_64 running kernel 5.15 (./haveged --Foreground) and it works fine there.

Thanks a lot
Jirka

[eworm-de]

Now does it make sense to revert cef1d42?

[frostschutz]

It seems to work fine for me.

Without haveged:

[    0.074495] random: get_random_u64 called from __kmem_cache_create+0x2a/0x4c0 with crng_init=0
[    1.353786] random: fast init done
[    3.428981] random: cryptsetup: uninitialized urandom read (4 bytes read)
[   27.874776] random: cryptsetup: uninitialized urandom read (32 bytes read)
[   32.867232] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   32.867418] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   32.867425] random: cryptsetup: uninitialized urandom read (64 bytes read)
[   42.252014] random: crng init done
[   42.252027] random: 5 urandom warning(s) missed due to ratelimiting

With haveged:

[    0.066190] random: get_random_u64 called from __kmem_cache_create+0x2a/0x4c0 with crng_init=0
[    1.476520] random: crng init done

(random fast init message is mysteriously missing. haveged itself spawns around the [ 0.8xxx] ~ [ 0.9xxx] mark.)

Using a simple initcpio hook (archlinux specific):

/etc/initcpio/install/haveged

#!/bin/bash

build() {
    add_binary "haveged"
    add_runscript
}

help() {
    cat <<HELPEOF
Haveged for early randomness and fast crng initialization.
HELPEOF
}

/etc/initcpio/hooks/haveged

#!/usr/bin/ash

run_earlyhook() {
    haveged
}

run_cleanuphook() {
    killall haveged
}

So nothing fancy, it simply starts haveged early initramfs and kills it late initramfs. haveged works its magic in the meantime. haveged as a service to be spawned again later on by the real init system.

[eworm-de]

(random fast init message is mysteriously missing. haveged itself spawns around the [ 0.8xxx] ~ [ 0.9xxx] mark.)

It is not mysteriously. As enough entropy is available fast init is skipped completely in favor of complete crng init.

Using a simple initcpio hook (archlinux specific):

[snipped initcpio hooks]

So nothing fancy, it simply starts haveged early initramfs and kills it late initramfs. haveged works its magic in the meantime.

I could add something like this in the package....
But wondering if it makes sense to add a new switch --once or --early. It could inject entropy once, then terminate itself.

haveged as a service to be spawned again later on by the real init system.

With the switch from above it would be possible to keep the current service haveged.service as is, including the condition on kernel version.
Adding a new service haveged-once.service using the new switch would allow to add a service in systemd-enabled initramfs image.

[klausenbusk]

I'm a bit puzzled why the jitter entropy in the kernel (merge commit, commit, LKML) isn't working. Is there anything exotic about your setup @frostschutz?

Edit: Is it blocking boot @frostschutz?

[jirka-h]

it would be possible to keep the current service haveged.service as is, including the condition on kernel version.

👍 I like it!

I have added a new switch --once and I have kept haveged.service unchanged. Could you please give it a try? If it works fine, I will release a new version.

Thanks a lot!
Jirka

[jirka-h]

commit 98ead65

[frostschutz]

Is there anything exotic about your setup @frostschutz?

My desktop is a standard arch linux install with a custom encryption hook since I have more than just the one LUKS device. But in the end it still runs a standard cryptsetup open ... and waits for me to enter passphrase. And that works fine, except crng init simply never happens until there is actually activity from my end, so the crng init after 100 seconds on bare metal is because I wasn't typing anything.

My virtual server is a little exotic in that its encrypted and uses cryptsetup very early to check or change the passphrase. So it actually wants to use the random device very early, hence the warnings about cryptsetup reading random before crng is fully initialized.

It is not blocking in either case, so maybe this is for cosmetics only, it just doesn't give me a good feeling to see such warning messages or late initializations.

The kernel is very conservative about randomness/entropy (for good reasons, I'm sure) but even if the kernel 'fixes' it, I still want to keep using haveged... I also have other things in place like an early random seed (the systemd random seed service runs long after initramfs is done so maybe a little late) but I did not use those in the above tests.

Basically however good the kernels random implementation is or will be, I still feel that userspace should mess with it just a little regardless, and for that purpose I'd love to continue using haveged, both in initramfs as well as a service that just keeps running indefinitely.

So this is my personal feeling but I'd still love the kernel 5.6 condition to go away, after all I installed the thing and enabled the service because I want it to run and do something. ;-) I'm sure there will be people who don't need it but they simply won't install or activate it either?

I can patch the service file locally or just install my own, but I can't patch haveged itself, so @jirka-h thanks a lot for doing that especially between the years — I really appreciate it.