Remix.run Logo
arp242 7 hours ago

Cheers; I guess I should have checked the coreutils implementation; I kind of just assumed it has one implementation instead of being a compile option :embarrassed-emoji:

I also have an Arch machine where it does link to libcrypto, and it seems roughly identical (or close enough that I don't care, this is a live server doing tons of $stuff so has big error bars):

  md5sum              1.58s user 0.31s system 98% cpu 1.908 total
  ~/verify -a md5     1.59s user 0.13s system 99% cpu 1.719 total
  
  sha256sum           0.71s user 0.12s system 99% cpu 0.840 total
  ~/verify -a sha256  0.74s user 0.12s system 99% cpu 0.862 total
Still wish it could do multi-core though; one reason I looked in to this is because I wanted to check 400G of files and had 15 cores doing nothing (I know GNU parallel exists, but I find it hard to use and am never quite sure I'm using it correctly, so it's faster to write my own little Go program – especially for verifying files).
collinfunk 6 hours ago | parent [-]

Interesting, there must be something wrong here. Here is a benchmark using the same commit and default options other than adjusting '--with-openssl=[yes|no]':

  $ dd if=/dev/random of=input bs=1000 count=$(($(echo 10G | numfmt --from=iec) / 1000))
  10737418+0 records in
  10737418+0 records out
  10737418000 bytes (11 GB, 10 GiB) copied, 86.3693 s, 124 MB/s
  $ time ./src/sha256sum-libcrypto input 
  b3e702bb55a109bc73d7ce03c6b4d260c8f2b7f404c8979480c68bc704b64255  input

  real 0m16.022s
  $ time ./src/sha256sum-nolibcrypto input 
  b3e702bb55a109bc73d7ce03c6b4d260c8f2b7f404c8979480c68bc704b64255  input

  real 0m39.339s
Perhaps there is something wrong with the detection on your system? As in, you do not have this at the end of './configure':

  $ grep -F 'HAVE_OPENSSL_' lib/config.h
  #define HAVE_OPENSSL_MD5 1
  #define HAVE_OPENSSL_MD5_H 1
  #define HAVE_OPENSSL_SHA1 1
  #define HAVE_OPENSSL_SHA256 1
  #define HAVE_OPENSSL_SHA3 1
  #define HAVE_OPENSSL_SHA512 1
  #define HAVE_OPENSSL_SHA_H 1
arp242 4 hours ago | parent [-]

Sorry, I meant "roughly identical [to my Go program]", not "roughly identical [to the version without OpenSSL]". The ~/verify binary is my little Go program that is ~4 times faster on my Void system, but is of roughly equal performance on the Arch system, to check that coreutils is not slower than Go (when using OpenSSL). Sorry, I probably didn't make that too clear.