> "let alone with AI" what's falling into the AI category here perhaps is the key question, since microseconds counts, and LLM are very slow!

LLMs (Large Language Models) are far from the only type of AI around. It's a pretty broad field, and there are real-time AI systems, for example, self-driving cars, which have the response times you're thinking of. [1]

[1] https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Mod...

hence: "what's falling into the AI category here perhaps is the key question"

YOLO is a good example for something that can work.

https://docs.ultralytics.com/models/yolov8/