Basically this exists to solve one of my biggest pain-points with dataframe libraries and big data. It's basically impossible to lazily download data on S3 with polars unless you're processing it strictly sequentially, and even then, it won't work with arrow and numpy data.

This allows you to "download" a multi-gigabyte file on S3 into a Polars DataFrame in <100ms.

# Demo

    alloc = demandmap.S3Alloc(
        "./cache.bin",
        # number of blocks
        capacity=512,
        # one megabyte block (per request chunk size)
        block_size=1048576
    )

    buf1 = alloc.get(S3_PATH)
    # Big file
    assert buf1.nbytes > 400000000
    # But this takes ~100ms
    df = pl.DataFrame([buf1])

I'd really like to build a cross platform user faulting library covering Linux and Windows too, because nearly everyone who's touched a dataframe has had this exact problem.