I am currently working on Clippy Vision, which is a fully local screen watcher that watches your screen continuously and builds context over time so you don't have to re-explain context like you do have to in case of Claude or ChatGPT.

Currently, its reactive, i.e a simple ReAct loop which calls function to retrieve data from memory and answers question when asked. Everything runs locally (Ollama + a local vision model, SQLite for storage), nothing leaves the machine.

Right now,I am planning to add a skill layer on top of the base watcher. So instead of it just being reactive, it acts proactively, for eg: spotting some mistakes in my code that I may have missed, or when you see X do Y types, and so on.

I am actively building this project. Version 1 is out already. source code: https://github.com/protocorn/clippy-vision