Here's my impressions of your algorithm:

1. read each site

2. rent a 4090 with https://vast.ai to run vllm

3. let llm model invent its own category and tag names freely

4. save 1KB of metadata each

  a. a small local language model that reads each one and writes a name, two or three sentences, a category, and a handful of tags.

The technical details are on another page: https://alexmorleyfinch.github.io/marlin/history/v1/article/...

Your impressions seem about right, but there are a few control steps it seems.

They really needn't have specified "in a weekend" cause yeah we can tell.

Since when has low effort become a selling point anyhow?