> I just wish there was some kind of ACID test suite for XMPP or something else to reproducibly validate spec compliance. Maybe a test server or similar as a reference implementation. This way client or server maintainers would have to run their programs against the official test server to increase their compliance stats.

This is exactly what the Compliance Suits are for, and the XMPP Software Fundation is taking care of telling all the clients what they misses directly on the official website, for example: https://xmpp.org/software/movim/

▲

MarsIronPI 4 hours ago | parent | next [-]

There is the XMPP Compliance Tester[0] by the author of Conversations. It does a good job at testing servers. On the client side I'm not aware of any kind of benchmark.

[0]: https://codeberg.org/iNPUTmice/caas

▲

cookiengineer 8 hours ago | parent | prev [-]

My point is about why clients like AstraChat can be listed with "Advanced" in the overview, but then in the details page it has nothing. See https://xmpp.org/software/astrachat-xmpp-client/

This should not be allowed.

▲

edhelas 7 hours ago | parent [-]

Because the declaration file of the clients says that it is actually compatible with everything in this section.

You can't run scripts on all the XEPs declared, some of them are purely redaction or bound to specific UI/UX behaviors. This is based on trust that the developers actually implemented things as stated.

	▲	saghm 5 hours ago \| parent [-]
		Not being able to automate something is not the same as not being able to verify at all. It sounds like the parent commenter is arguing that at least some of the clients listed are not worthy of this trust because (either intentionally or due to developer error) they don't actually hold up to scrutiny. Obviously they're just one person and their opinion might not be representative but it's hard to argue that if some random user is expected to have enough time to try out various clients and figure out which ones work or don't that the official people in charge of making the recommendations of clients should probably be able to find the time to as well even if it's just a volunteer that they, well, you know...trust.