| ▲ | drob518 7 hours ago |
| From the article: > I was later surprised all the real world find implementations I examined use tree-walk interpreters instead. I’m not sure why this would be surprising. The find utility is totally dominated by disk IOPS. The interpretation performance of find conditions is totally swamped by reading stuff from disk. So, keep it simple and just use a tree-walk interpreter. |
|
| ▲ | Someone 5 hours ago | parent | next [-] |
| Is it truly simpler to do that? A separate “command line to byte codes” module would be way easier to test than one that also does the work, including making any necessary syscalls. Also, decreasing CPU usage many not speed up find (much), but it would leave more time for running other processes. |
| |
| ▲ | drob518 5 hours ago | parent | next [-] | | If it was easier to interpret byte codes, nobody would use a tree-walk interpreter. There’s no performance reason to use a tree-walk interpreter. They all do it because it’s easy. You basically already have the expression in tree form, regardless of where you end up. So, stop processing the tree and just interpret it. | |
| ▲ | maxbond 5 hours ago | parent | prev [-] | | File operations are a good candidate for testing with side effects since they ship with every OS and are not very expensive in a tmpfs, but you don't have to let it perform side effects. You could pass the eval function a delegate which it calls methods on to perform side effects and pass in a mocked delegate during testing. |
|
|
| ▲ | chubot 6 hours ago | parent | prev [-] |
| Yeah that's basically what was discussed here: https://lobste.rs/s/xz6fwz/unix_find_expressions_compiled_by... And then I pointed to this article on databases: https://notes.eatonphil.com/2023-09-21-how-do-databases-exec... Even MySQL, Duck DB, and Cockroach DB apparently use tree-walking to evaluate expressions, not bytecode! Probably for the same reason - many parts are dominated by I/O, so the work on optimization goes elsewhere And MySQL is a super-mature codebase |
| |
| ▲ | drob518 4 hours ago | parent [-] | | I was just reading a paper about compiling SQL queries (actually about a fast compilation technique that allows for full compilation to machine code that is suitable for SQL and WASM): https://dl.acm.org/doi/pdf/10.1145/3485513 Sounds like many DBs do some level of compilation for complex queries. I suspect this is because SQL has primitives that actually compute things (e.g. aggregations, sorts, etc.). But find does basically none of that. Find is completely IO-bound. | | |
| ▲ | hxtk 2 hours ago | parent [-] | | Virtually all databases compile queries in one way or another, but they vary in the nature of their approaches. SQLite for example uses bytecode, while Postgres and MySQL both compile it to a computation tree which basically takes the query AST and then substitutes in different table/index operations according to the query planner. SQLite talks about the reasons for each variation here: https://sqlite.org/whybytecode.html | | |
|
|
