| ▲ | TornadoVM 7 Enables GPU-Native Workloads from Java Applications(infoq.com) |
| 65 points by pjmlp 2 days ago | 12 comments |
| |
|
| ▲ | mikepapadim 3 hours ago | parent | next [-] |
| https://github.com/beehive-lab/TornadoVM this is main repo. You can use it with any JDK21+ and Nvidia or Apple hardware and their Nvidia libs or MLX libs. |
| |
| ▲ | Akseleu_omirbay 2 hours ago | parent [-] | | And what about TPU support? Is it compatible with Google TPUs or strictly limited to gpu backends like CUDA and Metal? | | |
|
|
| ▲ | the_arun 3 hours ago | parent | prev | next [-] |
| Does this mean we can use Java to build AI infrastructure? |
| |
| ▲ | vlowrian 3 hours ago | parent | next [-] | | Or an abstract AI infrastructure factory? | | | |
| ▲ | mike_hearn 27 minutes ago | parent | prev [-] | | Yes (or any JVM lang). There is also this: https://qxotic.ai/ ... which is a single JAR that does SIMD accelerated inferencing of local models, dependency free. Everything is written in Java from the tokenizer up. It can also use Metal on macOS. Small models can be surprisingly powerful these days and all you need is a fast CPU. Remarkably, it is significantly faster than llama.cpp on some tasks like prefill (decode is memory bandwidth constrained so language hardly makes a difference). At least as long as you run on GraalVM (JVM or native image, both work). So you can actually beat C++ performance for AI in Java. It's impressive stuff. |
|
|
| ▲ | amir734jj 3 hours ago | parent | prev [-] |
| Basically JAX for Java instead of Python |
| |
