Remix.run Logo
aspenmayer 7 months ago

True, there are a lot of bots that use Internet Archive, which is probably the easiest to scrape. Maybe ask Jason Scott of Archive Team if he has any ideas for how to use IA and other archives for this purpose, and for ideas about how else to get this data?

I think Instapaper was another solution in this space that may have the info you want.

Maybe ask on some data hoarder subreddits about how to find new content that’s relevant to your interests with existing social proof?

I can see how the data from Pocket would have made that a lot easier for you, but finding a quick solution may be difficult. I think Apple News has a bit of social components around surfacing popular content, but that is not the same as user generated content indicating interest in a specific site, which is your goal.

Are you familiar with MetaFilter? There a community that might have some insight into your question, as they’re like HN but somewhat more crunchy and broadly less technical, but very human. Asking around other communities, you might find some suggestions.

Please let me know if you find a solution because this is an interesting problem, and I would probably be just as interested in the solution.