Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ cesarb 14 hours ago

It's hard to explain without going into a few low-level details of PCI Express, but let me try.

Most PCI devices expose some memory and/or I/O ports to the CPU. That memory (or I/O ports) is mapped to somewhere in the address space visible to the CPU. Besides the memory and I/O ports, all PCI devices also expose a separate set of configuration registers; among these registers, there are the Base Address Registers (BARs), which configure where the memory or I/O ports is mapped.

Here's an example output from "lspci -vv" for a GPU:

        Region 0: Memory at 7c00000000 (64-bit, prefetchable) [size=8G]
        Region 2: Memory at 7e00000000 (64-bit, prefetchable) [size=256M]
        Region 4: I/O ports at f000 [size=256]
        Region 5: Memory at fca00000 (32-bit, non-prefetchable) [size=1M]
        Expansion ROM at fcb00000 [disabled] [size=128K]
Note that regions 0 and 2 are above the 4GB addressable by old 32-bit CPUs. To be compatible with these old CPUs, this card and many others like it allow the firmware (and/or the operating system) to choose not only where the memory is mapped, but also its size. We can see this in the same "lspci -vv" output for this GPU:

        Capabilities: [200 v1] Physical Resizable BAR
                BAR 0: current size: 8GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
                BAR 2: current size: 256MB, supported: 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB
Older systems which do not understand this extended capability will still treat these regions as fixed size, probably with the first size in this list (256MB for region 0, 2MB for region 2). Newer systems can tell the device to "resize" the BAR to a bigger size, which obviously needs the first region to be placed above the 4GB barrier since it's too big.

Why is this useful? This particular GPU has 8GB of VRAM; it's quite obvious that region 0 is a direct view into that VRAM. When using the maximum BAR size, the CPU can directly read and write anywhere into the VRAM; when using a smaller BAR, the CPU can only see a small window into the VRAM, and has to use less direct methods to access it.

(As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)

▲ChocolateGod 3 hours ago | parent [-]

> (As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)

IIRC Linux doesn't need resizable bar enabled in the BIOS since the kernel will resize the bar if supported by the GPU, Windows however relies on the UEFI doing it which is where it being enabled in the BIOS is needed.