Remix.run Logo
vlovich123 2 days ago

I think this says more about the terrible memory bandwidth & limited compute of the Intel mobile CPUs than about the positive speed of SSDs. Here's an 13900K 64 GiB with a SN850X SSD LUKS encrypted ext4. On my machine RAM compression is still faster. There's also various overheads in this test that make it not a 100% representative sample although I'm not sure how big the divergence is (namely zram swap doesn't have a filesystem and it's deep within the memory management code and not using O_DIRECT).

Basic memory bandwidth test:

    $ dd if=/dev/zero of=/dev/null bs=10M count=7000
    7000+0 records in
    7000+0 records out
    73400320000 bytes (73 GB, 68 GiB) copied, 1.44856 s, 50.7 GB/s
Read test $ dd if=random.bin of=/tmp/random.bin conv=fdatasync bs=10M iflag=direct oflag=direct 2500+0 records in 2500+0 records out 26214400000 bytes (26 GB, 24 GiB) copied, 9.09728 s, 2.9 GB/s

Write test

    $ dd if=/tmp/random.bin of=tmp.bin conv=fdatasync bs=10M iflag=direct oflag=direct
    2500+0 records in
    2500+0 records out
    26214400000 bytes (26 GB, 24 GiB) copied, 53.9548 s, 486 MB/s
Not sure why that disk write test was suddenly so bad.

    $ df -h /mnt
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/zram0       30G  2.1M   28G   1% /mnt
Write test to lz4-compressed ZRAM:

    $ sudo dd if=/tmp/random.bin of=/mnt/tmp.bin conv=fdatasync bs=10M iflag=direct oflag=direct
    2500+0 records in
    2500+0 records out
    26214400000 bytes (26 GB, 24 GiB) copied, 7.97006 s, 3.3 GB/s
Read test:

    $ dd of=/tmp/random.bin if=/mnt/tmp.bin conv=fdatasync bs=10M iflag=direct oflag=direct
    2500+0 records in
    2500+0 records out
    26214400000 bytes (26 GB, 24 GiB) copied, 5.16566 s, 5.1 GB/s
fpoling 2 days ago | parent [-]

What is random.bin? I was testing with a json dataset that compresses by factor like 2.5 with zram. But if random is incompressible, then zram does not write compressed data but rather the original resulting in much faster read speed.

Also on your SSD do you have logical 4K sector or 512 byte sectors? If the latter, then Linux distros defaults to 512 LUKS sectors on them resulting in much slower performance especially with writes.

I always ensure that LUKS sectors are 4K even if SSD reports 512 bytes and does not allow to change that to 4K like Samsung 9* series.

vlovich123 a day ago | parent [-]

4k sectors. I don’t have a 25gib json file. Where can I get/generate the dataset you were using?

fpoling a day ago | parent [-]

The dataset is internal, but if you get a bunch of web pages from wikipedia and wrap them into HTML then it give a rough idea.

With 4K LUKS sectors the write speed is too low for a modern SSD. Check that LUKS use a fast implementation. I have:

  $ /usr/sbin/cryptsetup benchmark
  ...
  #     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b       664.1 MiB/s      2615.9 MiB/s
... aes-xts 512b 2708.8 MiB/s 2986.0 MiB/s

and my LUKS setup uses aes-xts 512b.