FYI, your "8‿64 •bit._cast -⟜@ raw" in bqn/util.bqn results in passing to •bit._cast integers in the range 0..255, whereas it expects integers ¯128..127 (and does arbitrary platform-specific things on integers outside that for the int16→int8 narrow); removing the "-⟜@" makes it work properly, and 2.5-5x faster while at it from not going to intermediate int16 (allowing the bitcast to be O(1) by itself). (•FBytes → •file.MapBytes also probably improves things, at the cost of the obvious mmap problem of issues if the file changes, as BQNs arrays are immutable). This is also what caused the problem at https://github.com/ClickHouse/ClickBench/pull/939/changes/40... (there's been some discussion about what to do about invalid •bit._cast before, considering it's rather perf-sensitive; it's the only place in CBQN with such quiet very-wrong results on invalid inputs)

The default CBQN "make o3" on x86-64 also results in it only using SSE2 (utilizing function multiversioning is on the ever-infinite TODO list, though somewhat-low on it considering it's strictly-unnecessary in any specific situation; there's also AVX-512 usage on a branch, but mostly only AVX2 on mainline; and no arm SVE)

That all said, CBQN doesn't currently do any loop fusion, so being significantly-slower for sequences of operations over larger-than-cache arrays would kinda just be expected. BQN also just isn't particularly intended for database work anyway.

(didn't look much at the specific query impls, though "Pair" in utils.bqn is at least an overlong version of "Pair ← ⋈¨"; and some if not all of those Pairs would be better as "≍˘" to avoid nested arrays and ensuing pointer chasing; and, of course, if some of the columns are bools/int8/int16/int32, it'd be beneficial to store & load them as such instead of float64)

https://github.com/l-labs/db-benchmark/tree/main/l

there's one in the footer and agree.

that is fast! Are you going to do the STAC-M3 benchmarks? Kx use them as evidence that they are the fastest, which is a fair claim as nobody else participates in that particular test :-)

ah. cheeky.

yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)

Fair enough. Without seeing the code itself, one can only judge the book by its cover. I'm interested, but maybe not quite the target audience.

Not a web dev - but have some experience in benchmarking these types of workloads e.g. https://www.mcobject.com/press/november19-2014/

L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs

If there are others (will do ClickBench) they'll go there as well

and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/

shoutout to bryan @bcantrill who I think explained K best at https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 and https://queue.acm.org/detail.cfm?id=1531242

Algol?

Did you mean APL?

That’s fair: The most fair-to-submitter reading of your comment is my concerns are unfounded. If you didn’t intend that, that’s fine.

For moderators, I’d suggest that for the community, it’s spam. That’s one example, there are many more like it. The individual statement may be defensible but is still pseudoscientific language. This sort of content is a massive burden to community. Unanswerable anrguments about AI writing, whether the ability for an individual to have a parse-able reading is the same as writing being parable. The net effect is negative experiences for many and copy-editing for someone who did not do copy editing.

Thank you! I'm always finding new rules, even 16 years in :) Much appreciated when it's pointed out.

It attracts negative energy via unanswerable questions and downvotes if you assert it is, alas.