Remix.run Logo
tnelsond4 3 hours ago

I'm always inspired by SQLite. Overall I like it, but if you're not doing writes it's really overkill.

So I made a format that will never surpass SQLite, except that it's extremely lighter and faster and works on zstd compressed files. It has really small indexes and can contain binaries or text just like SQLite.

The wasm part that decompresses and reads and searches the databases is only 38kb (uncompressed (maybe 16kb gzipped)). Compare that to SQLite's 1.2mb of wasm and glue code it's 3% the size but searching and loading is much faster. My program isn't really column based and isn't suitable for managing spreadsheets, but it's great for dictionaries and file archives of images and audio.

I ported the jbig2 decoder as a 17kb wasm module, so I can load monochrome scans that are 8kb per page and still legible.

https://github.com/tnelsond/peakslab

SQLite is very well engineered, PeakSlab is very simple.

giza182 an hour ago | parent | next [-]

Perhaps a dumb question, but how do you get data into it if you’re not doing writes

tnelsond4 an hour ago | parent [-]

Generate it one time from a source tsv file or folder of media.

zoky 2 hours ago | parent | prev [-]

something something XKCD competing standards something something

tnelsond4 2 hours ago | parent [-]

Believe me, I tried sticking to SQLite or aard2 or stardict, they just were fundamentally inadequate with no good pwa cross platform tooling.

bbkane an hour ago | parent [-]

Does this remain true now that SQLite has a WASM build?

tnelsond4 an hour ago | parent [-]

Yes, because originally when I started PeakSlab it used the SQLite wasm build.