Remix.run Logo
exabrial 4 hours ago

not... yet! Speed actually was a byproduct hilariously. Compiled models are definitely lightning fast, likely the fastest they could ever be on the JVM, because the tree is directly encoded as bytecode; represented lots of Opcode.IFGT-like comparisons. The JVM's JIT Compiler will have a blast with these code paths.

Petrify will also be order of magnitude kinder to your Garbage Collector, which will increase performance in high-throughput situations. You're also not loading 10 gazillion classes, as your models are directly represented as a first-class Java Class.

The real goal here was the getting rid of dependencies! While thankful for the incredible (and free) work of the authors of the onnxruntime for Java, the primary onnxruntime jar a boat anchor; weighing is 90mb+ just by itself, not counting any of its dependencies.

Once you compile your models with Petrify, you have exactly one 6.9kb jar as a dependency essentially just carries the Fossil interface as an entry point to call your model. I licensed that jar ASL2.0 for maximum compatibility in a corporate environment.