Remix.run Logo
burntsushi 14 hours ago

Can you provide a concrete example where that's faster? ripgrep should generally already be approximating `git ls-files` by respecting gitignore.

Also, `-uu` tells ripgrep to not respect gitignore and to search hidden files. But ripgrep will still skip binary files. You need `-uuu` to also ignore binary files.

I tried playing with your `rgg` function. First problem occurred when I tried it on a checkout the Linux kernel:

    $ rgg APM_RESUME
    bash: /home/andrew/rust/ripgrep/target/release/rg: Argument list too long
OK, so let's just use `xargs`:

    $ git ls-files -z | time xargs -0 rg APM_RESUME
    arch/x86/kernel/apm_32.c
    473:    { APM_RESUME_DISABLED,  "Resume timer disabled" },
    include/uapi/linux/apm_bios.h
    89:#define APM_RESUME_DISABLED  0x0d

    real    0.638
    user    0.741
    sys     1.441
    maxmem  29 MB
    faults  0
And compared to just `rg APM_RESUME`:

    $ time rg APM_RESUME
    arch/x86/kernel/apm_32.c
    473:    { APM_RESUME_DISABLED,  "Resume timer disabled" },

    include/uapi/linux/apm_bios.h
    89:#define APM_RESUME_DISABLED  0x0d

    real    0.097
    user    0.399
    sys     0.588
    maxmem  29 MB
    faults  0
So do you have an example where `git ls-files -z | xargs -0 rg ...` is faster than just `rg ...`?
oever 13 hours ago | parent | next [-]

A checkout of my repository [0] with many pdf and audio files (20GB) is slow with -u. These data files are normally ignored because 1) they are in .gitignore and 2) they are binary.

The repository contains CI files in .woodpecker. These are scripts that I'd normally expect to be searching in. Until a week ago I used -uu to do so, but that made rg take over 4 seconds for a search. Using -. brings the search time down to 24ms.

    git ls-files -z | time xargs -0 rg -w e23
    40ms

    rg -w. e23
    24ms

    rgg -w e23
    16ms

    rg -wuu e23
    2754ms
To reproduce this with the given repository, fill it with 20GB of binary files.

The -. flag makes this point moot though.

[0] https://codeberg.org/vandenoever/rehorse

burntsushi 13 hours ago | parent [-]

Oh I see now. I now understand that you thought you couldn't convince ripgrep to search hidden files without also searching files typically ignored by gitignore. Thus, `git ls-files`.

Yes, now it makes sense. And yes, `-./--hidden` makes it moot. Thanks for following up!

EnPissant 6 hours ago | parent | prev [-]

I don't think this is the same thing as using gitignore.

It will only search tracked files. For that it can just use the index. I would expect the index to be faster than looking at the fs for listings.

burntsushi 5 hours ago | parent [-]

I was extremely careful with my wording. Re-quoting, with added emphasis:

> ripgrep should generally already be approximating `git ls-files` by respecting gitignore.

See also: https://news.ycombinator.com/item?id=45629515

EnPissant 4 hours ago | parent [-]

I'm just trying to be helpful, not call you out.

I've implemented gitignore aware file scanning before, and it was slower than git native operations when you only care about tracked files.

It's the speed that is the part I was speaking too, not the semantics.