| ▲ | JIT Compiling Code in 5μs(malisper.me) |
| 80 points by zX41ZdbW 6 hours ago | 46 comments |
| |
|
| ▲ | MaxBarraclough 3 hours ago | parent | next [-] |
| Reminds me of the 2024 blog post Look ma, I wrote a new JIT compiler for PostgreSQL [0]. Both articles lament that Postgres's LLVM-based JIT [1] takes a while to generate code. > The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile. That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM. [0] https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c... , discussed: https://news.ycombinator.com/item?id=39742916 [1] https://www.postgresql.org/docs/current/jit-reason.html |
| |
|
| ▲ | mgaunard 2 hours ago | parent | prev | next [-] |
| The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions. By not using LLVM, you're missing all the optimizations it does. |
| |
| ▲ | the-lazy-guy 13 minutes ago | parent | next [-] | | This is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline). LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time. | |
| ▲ | mort96 30 minutes ago | parent | prev | next [-] | | A non-optimizing compiler is a real compiler. | |
| ▲ | IshKebab 20 minutes ago | parent | prev [-] | | This absolutely is real JIT compilation. Copy and patch is a very well known JIT compilation technique. |
|
|
| ▲ | agnishom an hour ago | parent | prev | next [-] |
| I recommend Russ Cox's articles on implementing a regex engine: https://swtch.com/~rsc/regexp/ It is very relevant |
|
| ▲ | Gamer_S4lyer 25 minutes ago | parent | prev | next [-] |
| Nice |
|
| ▲ | glum64 4 hours ago | parent | prev | next [-] |
| Uhm, Common Lisp, where JIT is not only available but is also manageable: the programmer can decide what deserves to be compiled and what does not. Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...). |
| |
| ▲ | clbrmbr 2 hours ago | parent | next [-] | | is an xtensa lx7 (esp32-s3) target available that does not use llvm? | |
| ▲ | fweimer an hour ago | parent | prev [-] | | Not in typical builds of SBCL: all code is compiled before evaluation. |
|
|
| ▲ | ligarota an hour ago | parent | prev | next [-] |
| Tcc be like |
|
| ▲ | hamilyon2 3 hours ago | parent | prev | next [-] |
| It uses copy-and-patch compilation to archive that |
|
| ▲ | glenjamin 4 hours ago | parent | prev | next [-] |
| pgrust sounds very interesting, but with the deep changes there’s no viable path to upstream it - is the end goal to be robust enough that it’ll get wide adoption? |
| |
| ▲ | FiberBundle 2 hours ago | parent [-] | | Is it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers. |
|
|
| ▲ | varjag 2 hours ago | parent | prev | next [-] |
| There’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” I think that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that. |
| |
| ▲ | asdfsa32 2 hours ago | parent [-] | | Anyone who thinks AI is good with writing code that is hard to write for the operator, not due to lack of basic software engineering know how but complexity of the domain, either has access to models beyond what is available to the public or is completely lost. I believe this because every time I use AI for domains that I consider myself above competent, if it is anything beyond UI components or a simple CRUD endpoints, I cringe at the quality of what it generates. This has made me to be extremely cautious of starting working in a new domain with AI if I want anything beyond throw away quick hacks or junk, shy of quick bug fixes perhaps. | | |
| ▲ | vouwfietsman 12 minutes ago | parent | next [-] | | > I cringe at the quality of what it generates. Besides all the other mentioned points, I think a good remaining less-discussed point is that quality in software has always been in the eye of the beholder. You may very well see the AI output as low quality, I may not, and its not necessarily clear who is right or wrong, because there was always little precedent in objectively evaluating code quality. This is a long standing issue, and was never resolved before AI happened, and the coming of AI has not really changed things, except that AI is under a magnifying glass obviously. How do we objectively measure the quality of code? There's some general consensus on things, but surprisingly little is true professional agreed upon consensus. If I can make up a figure, I would guess 95% of software engineering quality rhetoric, and craftmanship advice, is just strongly held opinions. This is not something I can prove, but if I look at the (still ongoing.....) debates on very basic ideas like clean code, and the reactions from also-great programmers like Carmack, Blow & Muratori, it is clear to me that there is little consensus on even the fundamentals of software design. If all these people can produce excellent working software while disagreeing on these fundamentals (of quality), it means we do not yet understand what the fundamentals are. | |
| ▲ | mgaunard an hour ago | parent | prev | next [-] | | Fable (and even Opus if kept tightly under reigns) does generate high-quality code even for highly complex tasks. It generally performs better if the tasks are broken done into small manageable pieces, and the person is actually reviewing and calling out problems, which usually requires the person to be a competent engineer in the problem domain to begin with. But yes, I have personally used it to build what the OP calls a JIT. I would usually write that by hand and it would take me one week. The AI does it in an hour. | | |
| ▲ | asdfsa32 an hour ago | parent | next [-] | | You can't make assertions about "quality" of code that was generated under an hour while it would have taken a human 40 hours, unless you put substantial amount of work into reviewing it. I used Fable on a Zephyr project with time sensitive code for LR-WPAN and it broke everything. Literally made the code worst to the point that the devices stopped connecting. | |
| ▲ | Mawr 19 minutes ago | parent | prev [-] | | Your second paragraph invalidates your first. If I need to be a domain expert anyway, the value of the tool goes down by orders of magnitude. Same if I need to first break the task down into pieces and keep reviewing all the output. That sounds to me like >80% of the work I'd need to do anyway. If I need to design and understand all of the code anyway, I might as well skip the whole process of repeatedly fixing the subpar-at-every-level LLM output and write it all myself. Personally, I've found the greatest value in asking for simple, easy to verify tasks, like wiring up APIs, generating boilerplate, etc. Anything remotely complex and the LLM cakes its pants. |
| |
| ▲ | grebc an hour ago | parent | prev | next [-] | | Three quarters of my CS class at university could barely code and/or understand code. That’s not a joke. A lot went on to be programmers professionally. And judging by the quality of closed & open source code I witness daily those figures from university accurately depict people’s capabilities. Now that said, if you can’t really code then using AI will be a godsend to said individuals. | | |
| ▲ | mgaunard an hour ago | parent | next [-] | | I'd say that is the problem we're observing. A lot of decent code is being written with weird inconsistencies, because it's actually written by AI driven by people who don't really understand what they're doing. the tell-tale mark of AI code is highly over-engineered local solutions to trivial problems that don't matter, or that were already solved better elsewhere and that no sane human would ever duplicate. | |
| ▲ | asdfsa32 an hour ago | parent | prev [-] | | Fair, but I fear that now even more people who can't code will code, and code that is not any better than what people who could barely code write. Growing cabbages starting to look more and more interesting. |
| |
| ▲ | IshKebab 20 minutes ago | parent | prev [-] | | When did you last try? They've improved a lot. Also often the difficulty with writing code is simply knowing where to start - getting past the blank page. AI can help a lot with that. Often there's a task where I've got kind of writers block, but you can ask AI to do it and suddenly it's like "ah yeah, sort of but actually that's not quite right we should do it this way". |
|
|
|
| ▲ | roschdal 4 hours ago | parent | prev [-] |
| JIT compilation is unsecure. |
| |
| ▲ | stevefan1999 an hour ago | parent | next [-] | | So what, are you willing to go away from von-neumann architecture where instructions are data and data are instructions, i.e. the instruction-data hominocity that underpins JIT compilation? Are you willing to go to a pseudo-Harvard architecture where the ability of JIT compiling is soft locked by other means like VM or strong code authentication or policy protection, which is what Apple is doing. Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT | |
| ▲ | asdfsa32 3 hours ago | parent | prev | next [-] | | You're entirely correct because JIT requires violating Write xor Execute security policy. This is the reason on iOS, it is limited to Apple shipped software. https://en.wikipedia.org/wiki/W%5EX | | |
| ▲ | codethief an hour ago | parent | next [-] | | GrapheneOS heavily restricts JIT usage, too: > - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support. > - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […] > - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […] https://grapheneos.org/features | |
| ▲ | shakna 3 hours ago | parent | prev [-] | | The wiki page mentions this is only a minor problem. Because everyone just writes, then switches and executes. | | |
| |
| ▲ | sebzim4500 3 hours ago | parent | prev | next [-] | | Maybe, but surely there are users who are willing to trust all users of their db instance. | | |
| ▲ | asdfsa32 3 hours ago | parent [-] | | The issue is that it restricts from locking-down and securing the system with Write xor Execute memory. So it has system wide implication. https://en.wikipedia.org/wiki/W%5EX | | |
| ▲ | PhilipRoman 3 hours ago | parent [-] | | W^X is typically per mapping, not per memory page and does not interfere with JIT compilation. | | |
| ▲ | asdfsa32 3 hours ago | parent [-] | | Sure, but it still means that the OS has to decide who is allowed to do it and to what extent. Sophisticated worms like Stuxnet would be much harder with strict W^X for example, since CVE-2010-2568 and the like would be much harder to execute. | | |
| ▲ | orf 2 hours ago | parent | next [-] | | > Sure, but it still means that the OS has to decide who is allowed to do it and to what extent It has to do that anyway? | | |
| ▲ | asdfsa32 an hour ago | parent [-] | | Only if it wants to allow Writable Memory to become Executable, or basically, allow JIT. |
| |
| ▲ | pjmlp 2 hours ago | parent | prev [-] | | Signed binaries with the proper assigned OS capabilities. | | |
| ▲ | asdfsa32 an hour ago | parent [-] | | Yes, but with JIT, you can't really verify what the application does upfront. That is the entire point. | | |
| ▲ | pjmlp an hour ago | parent [-] | | Capabilities are a way to control that, and the point being that only responsible proven applications get the certificate, hence how it all goes on iOS. | | |
| ▲ | asdfsa32 an hour ago | parent [-] | | You're making the assumption that "responsible" is something provable, but that is not the case, it is specially not easy to prove software is secure from tampering its behaviour. |
|
|
|
|
|
|
| |
| ▲ | dennis16384 4 hours ago | parent | prev | next [-] | | It is the core of ClickHouse for example, for many years. Is it secure enough in your opinion? | |
| ▲ | JackSlateur 2 hours ago | parent | prev | next [-] | | In rust, is jit equivalent to an "unsafe" block ? | | |
| ▲ | brabel 30 minutes ago | parent [-] | | Read the code in the post. Everything is written in unsafe Rust. The assembly itself knows no memory safety at all and is completely up to the programmer skill whether it can be trusted to not mess up. |
| |
| ▲ | pjmlp 2 hours ago | parent | prev [-] | | Machine code is insecure, we should all run interpreted code in a formally verified interpreter. Alternatively, only allow for the execution of cryptographly signed static linked binaries, this naturally includes the interpreter above. |
|