I had students build this kind of thing in 2020 by screenshotting the game and processing it with a standard OpenCV pipeline. No GenAI needed.

▲

throwaway894345 3 days ago | parent [-]

Thank you for educating me. How does OpenCV work from the perspective of recognizing things in an image? Is there some kind of underlying model there that learns what a target looks like or not?

▲

ModernMech 3 days ago | parent [-]

The way they did it, they were writing an aimbot. So the pipeline was:

- take a screenshot

- run massive skeletal detection on it to get the skeletons of any humanoids

- of those skeletons, pick a target closest to the player

- for that target, get coordinates of head node

- run a PID to control the cursor to where the head node is located

- move the camera one frame, repeat the whole process. If you can fit that pipeline to 16ms it can run in real time.

▲

throwaway894345 2 days ago | parent [-]

Wow, that's fascinating. Were they able to fit the whole thing inside the 16ms frame?

	▲	ModernMech 2 days ago \| parent [-]
		oh yeah, with little problem, especially with a GPU it's not hard at all.