A lot of discussion around RSS revolves around the format for the data/metadata (e.g. the Atom feud) but the real problem with it is this:

To consume an RSS feed you poll it. There are two polling speeds: too fast and too slow, and it's possible to be both at the same time.

Note the struggles of this Karen to turn RSS from a simple stateful protocol to a complex stated protocol, and she'll ban you if you ever reset your cache and rerun your fetcher because your cache got corrupted or you suspect it might have been corrupted.

http://rachelbythebay.com/w/2022/03/07/get/

You really want to have a stream of feed items and to be able to: (1) replay the whole stream all the way from or to the beginning and (2) query "what got added after time t?" and just get that. ActivityPub accomplishes this but people don't really like it. For Dave Winer it is all blub but even if he doesn't believe in the Fedi, he's on it.

I really like

https://superfeedr.com/

because it does all the polling for you and hits your webhook whenever a new feed item appears. My webhook is about 15 lines of Python running as a Lambda function that posts items to an SQS queue and my YOShInOn RSS reader just drains the queue at its convenience. The pricing at 10 cents/feed/month is a bargain for high volume feeds like MDPI, arXiv, and The Guardian [1] but unfortunately I can't really afford to subscribe to 2000 little blogs that post maybe once a week at that rate. I wish there were more Planets.

https://en.wikipedia.org/wiki/Planet_(software)

[1] AWS costs would be trivial in comparison even if it got out of the free tier

▲

tolerance 4 days ago | parent | next [-]

It’s a pity that this is the bottom-most comment and equally it’s a shame that the slur “Karen” made its way into the White lexicon spoiling an otherwise informative remark.

Correct me if I’m wrong, but is Winer’s somewhat recent effort with FeedLandⁱ any different from Planet?

ⁱ: https://feedland.org/?username=scripting

▲

PaulHoule 4 days ago | parent [-]

(1) Some people read from the bottom up

(2) I really am mad at Rachel for this and that's from someone who's been writing webcrawlers [1] since 1998, been an RSS innovator, and been responsible for complex systems when they fail.

(3) Maybe I am missing it but I don't see an actual feed for that URL, I see only an OPML file. Dave is really gay [2] for OPML files but I'm not because I still have to work to fetch all the items. Yet, visually the OPML file and blogroll look like a planet and you're not the first person who's pointed Dave's blogroll as a solution as opposed to the problem that I see it is.

(4) Looking at the head of the list I think "Daily Kos" and "404 Media" suck but that I already subscribe to many of them like "Arstechica" -- looking at the tail of the list I see there are gems that I'm not getting. If those things were getting aggregated topically it would please both me and Rachel.

[1] that don't crash your server

[2] in a good sense!

▲

tolerance 4 days ago | parent | next [-]

> If those things were getting aggregated topically it would please both me and Rachel.

I think I catch what you’re getting at and it reminds me of what seems to be a primary gripe with RSS—compromising an otherwise decent spec as far as I can tell—discoverability and curation.

I can’t grasp the more technical issues surrounding polling just yet so I can only intuitively get how it corresponds with the other issues. In spite of this, and leaning again on my intuition, I didn’t get a good feeling from Rachel’s blog post when I first came across it either.

And yeah I’m not sure how FeedLand works under the hood, I just matched Wikipedia's description of a planet with what I knew FeedLand looks like on the frontend.

I also know there’s been some “folksonomic" efforts for curating lesser-know feeds, but to be honest I’m not fond of what these networks of folks appear to be into. Compare your distaste for Daily Kos and 404 Media (which I share) to the vibe I’m trying to avoid on that front.

▲

trekz 3 days ago | parent | prev [-]

> I really am mad at Rachel

Don't know what you're so angry with Rachel about. I provide feeds for readers and many of them are way too greedy and frequently do hammer websites, often unneccessarily. Mix in all the rampant AI bots and you got a recipe for an extremely expensive server bill.

Getting angry at someone who's trying to keep their server costs down to provide you something for free is kind of weird.

	▲	PaulHoule 3 days ago \| parent [-]
		It's a 10,000 word rant at least but... - I've looked at the abyss of bankruptcy from server bills. I actually think it's a hell of a lot worse than Rachel does, I thought that 15 years ago, and I've suffered worse - The whole discussion around RSS has been naive in every respect from the very beginning, for instance Dave Winer thinking we care about him publishing a list of the tunes he listens to back when you couldn't actually listen to them (I'll grant that in the age of Apple Music, Spotify, YouTube in such things may have caught up with him.) There are never any systems thinkers just people who see it all through a keyhole and can't imagine at all that anyone else sees it another way. - To use a Star Wars analogy, Google is the evil empire that blew up up our planet with a Deathstar 10 years ago and now we're living on the wreckage in an asteroid belt. I see the AI upstarts as The Rebel Alliance that at the very least reset the enshittification cycle back to the beginning and create some badly needed competition. Opponents to A.I. crawlers are brainwashed/living in the matrix and defacto defending Google's monopoly and slamming the door to prevent the exits from enshittification that A.I. agents offer (e.g. they look at the content and drop out the ads!) They think they're the rebel alliance but are really the stormtroopers suppressing the resistance until the Deathstar charges up for the next shot. - Rachel's supposedly some high powered systems programmer or sysadmin or something but is just as naive as Winer. We're supposed to "just" use a cache. Funny, the reason why the web won out of every other communication protocol is that you can "just" use curl. curl by it's nature can't support cacheing because it's a simple program that just runs in one process and if you wanted to have a cache you'd have to have a complex locked data structure which would force a bunch of trade offs... like I am downloading something over my slow ADSL connection that takes two hours but I also need to clear my cache so do I force the download to abort or hang up the script that needs the cache cleared for two hours? curl is "cheap and cheerful" because you just don't have to deal with all the wacky problems for which "clear the cache" or "disable the cache" clears up like a can of Ubik. But in the age of "Vibe Coding" solutions that almost work are all the rage, except when you finally realize they didn't actually work you clear your cache and rerun and BAM you got banned by Rachel's blog because you hit it twice in 24 hours. Web browsers are some of the most complicated software in existence, ahead of compilers (they contain compilers) and system runtimes (they are a system runtime), right up there with operating systems (less having to know things that aren't in the datasheet to write device drivers at least.) For the first 15 years you could not trust the cache if you were writing web applications, somewhere around 2010's I finally realized you could take it for granted the cache works right. I guess implementations and the standards improved over time but all this complexity is some of the reason why it is just Google's world and we all live in it and there are just two big techs that can make a browser engine and one unaccountable and out-of-touch foundation. So I wish Rachel would just find an answer to the cost problems (Cloudflare R2?) or give up on publishing RSS or advocate activitypub rather than assume we care what she says enough to follow her rules for her blog without seriously confronting what a system-wide solution for the problems RSS tries to solve and the problems it poses.

▲

account42 4 days ago | parent | prev | next [-]

This is really not much of an issue if both sides implement HTTP caching (If-Modified-Since or Etags). Atom also adds pagination which allows you to keep all old items accessible to supporting readers while cutting down the main feed to just the last entry. The little bandwidth a well managed feed takes is really not worth giving up the ability to host the feed statically which the overly complex ActivityPub can't do.

▲

PaulHoule 4 days ago | parent [-]

You are talking about this sort of thing?

https://datatracker.ietf.org/doc/html/rfc5005

Http might be better in 2025 than it used to be but historically the cache is as much a problem as it is a solution. That is, when you have heisenbugs the answers are frequently "clear the cache" [1] and "don't use the cache" [2] and it's often a better strategy to "rip and archive the whole site right now and come back in six months and make a fresh rip" if you're working on a large scale and can tolerate being up to six months out of date.

In general database-backed sites struggle to implement if-modified-since since a fully correct implementation has to traverse the graph of all database objects that are looked up in the request which costs about as much as… the request. Your cache needs a cache and you can make it work at the system level if you have a modified date cache and always invalidate it properly. If you are doing that you might as well materialize a static web site fronting your site the way the Wordpress supercache works —- then you’ll find your site crashing a lot less!

I’ll admit ActivityPub is complex but http caching is complex in a particularly poisonous way in that there are so many alternate paths. An ActivityPub system (client+server) could be 100% correct but an http-based system might not have a clear line around it and might never be quite right. A stateless system could run for 10 years without trouble, it might take you 10 years to realize a stateful system was feeding you corrupted data.

[1] fix it... for now

[2] ... and they're no longer part of your life

	▲	account42 3 days ago \| parent [-]
		> In general database-backed sites struggle to implement if-modified-since since a fully correct implementation has to traverse the graph of all database objects that are looked up in the request which costs about as much as… the request. That's only true if your implementation favors editing performance over GET request performance ... in which case you get what you ordered. If your feed requires any database connections then that's a choice you made. I disagree that HTTP caching is complex. The basic primitive are: - An indication until when the returned document can be used without checking for updates. - A way to check if a cached document is still valid (based on either modification time or id). This provides everything you need to implement an efficient feed reader.

▲

Karrot_Kream 4 days ago | parent | prev | next [-]

Do you know (or have an idea about public discussions indicating) why ActivityPub never ended up being used for feeds? AP seems to mostly be Mastodon and Lemmy but it obviously has a bunch of fixes for problems inherent in RSS, Atom, and OPML.

	▲	PaulHoule 4 days ago \| parent [-]
		I’ve seen a lot of Dave Winer saying his blub is good enough and abstractly that ActivityPub is too complex. There’s pixelfed too. Practically if I want to have an ActivityPub feed for my blog I might host in on Mastodon and if the character limit is too little you can have a server like https://writefreely.org/ If anything characterizes the RSS community it is a lack of imagination and rejection of the last 25 years of work in relevance, ML and UX. RSS readers are still failing with the same failing interfaces that failed 25 years ago. That said, the people who are still using it feel satisfied and other than Rachel that aren’t a lot of people who will try to stop you if you do try something ambitious. You just gotta poll poll poll and poll poll poll some more or pay somebody to poll poll poll for you.

▲

jonchang 4 days ago | parent | prev [-]

Her name is actually Rachel, not Karen.

▲

PaulHoule 4 days ago | parent [-]

I know. Perhaps I should have said "one of those misanthropic online personalities who kills 1000 contributions to open source you never knew you would have had because of their public bad attitude" and the fact that they have supporters is why Facebook wins and open systems lose.

	▲	DamonHD 4 days ago \| parent [-]
		Rachel is absolutely not like this in my experience. How rude!