| ▲ | csdreamer7 3 days ago |
| This means their servers are very old ones that do not support x86-64-v2. Intel Core 2 Duo days? https://developers.redhat.com/blog/2021/01/05/building-red-h... Think of how much faster their servers would be with one of those Epyc consumer cpus. I was about to ask people to donate, but they have $80k in their coffers. I realize their budget is only $17,000 a year, but I am curious why they haven't spent $2-3k on one of those Zen4 or Zen5 matx consumer Epyc servers as they are around under $2k under budget. If they have a fleet of these old servers I imagine a Zen5 one can replace at least a few of them and consume far less power and space. https://opencollective.com/f-droid#category-BUDGET Not sure if this includes their Librapay donations either: https://liberapay.com/F-Droid-Data/donate |
|
| ▲ | bayindirh 3 days ago | parent | next [-] |
| > This means their servers are very old ones that do not support x86-64-v2. Intel Core 2 Duo days? This is not always a given. In our virtualization platform, we have upgraded a vendor supplied VM recently, and while it booted, some of the services on it failed to start despite exposing a x86_64v2 + AES CPU to the said VM. Minimum requirements cited "Pentium and Celeron", so it was more than enough. It turned out that one of the services used a single instruction added in a v3 or v4 CPU, and failed to start. We changed the exposed CPU and things have returned to normal. So, their servers might be capable and misconfigured, or the binary might require more that what it states, or something else. |
| |
| ▲ | lucb1e 3 days ago | parent [-] | | A developer on the ticket writes: "Our machines run older server grade CPUs, that indeed do not support the newer SSE4_1 and SSSE3" | | |
| ▲ | bayindirh 3 days ago | parent [-] | | Ooh. They are at least ~15 years old, then. Maybe they have scored on some old, 4 socket Dell R815s. 48 cores ain't that bad for a build server. | | |
| ▲ | lucb1e 3 days ago | parent [-] | | It's kinda good they use such old systems, as the vast majority of pollution occurs during manufacturing of devices since we usually use them only a handful of years. Iirc the break-even point was somewhere around 25 years, as in, upgrading for energy efficiency then becomes worth it (source: https://wimvanderbauwhede.codeberg.page/articles/frugal-comp...). 15 goes a long way towards that! On the other hand, I didn't dig very deep into the ticket history now but it sounds like this could have been expected: it broke once already 4 years ago (2021), so maybe planning an upgrade for when this happens again would be good foresight. Then again, volunteers... It's not like I picked up the work as an f-droid user either | | |
| ▲ | NewJazz 3 days ago | parent | next [-] | | While I appreciate the sentiment, I think you may be misreading the "Emissions from production of computational resources" section of that link. It says for servers that 13-21 years is the break even for emissions from production vs consumption. The 25 year number is for consumer devices like phones and laptops. I would also argue that average load on the servers comes into play. | |
| ▲ | miladyincontrol 2 days ago | parent | prev [-] | | Moot point imo, no one says they have to buy new hardware. Used, affordable, but still much more modern hardware could still save them plenty on power usage and replace several systems with one. |
|
|
|
|
|
| ▲ | ignoramous 3 days ago | parent | prev | next [-] |
| > about to ask people to donate, but they have $80k in their coffers I'd still ask folks to donate. £80k isn't much at all given the time and effort I've seen their volunteers spend on keeping the lights on. From what I recall, they do want to modernize their build infrastructure, but it is as big as an investment they can make. If they had enough in their "coffers", I'm sure they'd feel more confident about it. It isn't like they don't have any other things to fix or address. |
| |
|
| ▲ | Timshel 3 days ago | parent | prev | next [-] |
| $2-3k ? That’s barely the price of a lower end Threadripper bare cpu not a full Epyc server ??? |
| |
| ▲ | wongarsu 3 days ago | parent | next [-] | | At our supplier $2k would pay for a 1U server with a 16 core 3GHz Epyc 7313P with 32GB RAM, a tiny SSD and non-redundant power. $3k pays for a 1U server with a 32 core 2.6GHz Epyc 7513 with 128GB RAM and 960GB of non-redundant SSD storage (probably fine for build servers). All using server CPUs, since that was easier to find. If you want more cores or more than 3GHz things get considerably more expensive. | | |
| ▲ | Timshel 3 days ago | parent | next [-] | | Yes but thoose are Zen 3 Milan cpu released in 2021 I believe. Not that they are bad and would not be way better than what they have, just that I though the parent was quite the optimist with his Zen4/Zen5 pricing. | | |
| ▲ | wtallis 3 days ago | parent [-] | | OP did say "consumer Epyc", so presumably referring to the parts using the AM5 socket. From a quick check on Newegg, it looks like barebones servers for that platform start at under $1000, to which you need to add CPU, RAM, and storage. So a $3000 budget to assemble a low-end Zen4/5 EPYC server is realistic: $570 for the 16-core EPYC 4565P, a few hundred for DDR5 ECC unbuffered modules, a few hundred for an enterprise SSD, and you have a credible current-gen server from readily available parts at retail prices, without any of the enterprise pricing and procurement hassle. | | |
| ▲ | BizarroLand 2 days ago | parent | next [-] | | I imagine they would need quite a few servers to replace their current setup. Then there's also the overhead of setting up and maintaining the hardware in their location. It's not just a "solve this problem for ~$2,000 and be done with it". I don't know the actual specs or requirements. Maybe 1 build server is sufficient, but from what I know there's nearly 4,000 apps on FDroid. 1 server might be swamped handling that much overhead in a timely manner. | | |
| ▲ | wtallis 2 days ago | parent [-] | | One server with today's tech can easily replace several servers that are 12+ years old. 4000 apps doesn't sound like a lot of work for one machine, unless you assume almost all of them are releasing new builds more than once a week. A 16-core CPU can rebuild a full Gentoo desktop OS multiple times a week. |
| |
| ▲ | csdreamer7 3 days ago | parent | prev [-] | | That was my intention; mATX AM5 parts. |
|
| |
| ▲ | speckx 2 days ago | parent | prev [-] | | Is that $2k/$3k for the year? | | |
| ▲ | wongarsu 2 days ago | parent [-] | | That's $2k/3k to get a box with fully assembled hardware delivered to your doorstep or to a DC of your choice. Space in your basement or the colo rack of a datacenter along with power, data and cooling is an expense on top. But whatever old servers they have are going to take up more space and use more power and cooling. Upgrading servers that are 5+ years old frequently pays for itself because of the reduced operating costs (unless you opt for more processing power at equal operating cost instead) |
|
| |
| ▲ | c0balt 3 days ago | parent | prev [-] | | Low end EPYC (16-24 cores) especially for older generations are not that expensive 800-1.2K ime. Less when in a second hand server. |
|
|
| ▲ | doublepg23 3 days ago | parent | prev | next [-] |
| Perhaps the servers run Coreboot / Libreboot? |
|
| ▲ | pclmulqdq 3 days ago | parent | prev | next [-] |
| I'm not even sure mainline Linux supports machines this old at this point. The cmpxchg16b instruction isn't that old, and I believe it's required now. |
| |
| ▲ | cwillu 2 days ago | parent | next [-] | | CMPXCHG8B is required as of a month or two ago, not 16B (i.e., the version from the 90's is now required) See https://lkml.org/lkml/2025/4/25/409 | |
| ▲ | csdreamer7 3 days ago | parent | prev [-] | | 32 bit Linux is still supported by the kernel and Debian, Arch, and Fedora still supports baseline x86_64. RHEL 8 is still supported and Ubuntu is still baseline x86_64 I believe for commercial distros. Not sure about SuSE. | | |
|
|
| ▲ | FirmwareBurner 3 days ago | parent | prev [-] |
| >they have $80k in their coffers but I am curious why they haven't spent $2-3k on one of those Zen4 or Zen5 matx consumer Epyc servers I would also like to know this. |
| |
| ▲ | pastage 3 days ago | parent | next [-] | | I would much rather they spent that on having the devs network and travel, the servers work. | | |
| ▲ | melodyogonna 3 days ago | parent | next [-] | | Why are the builds failing then? | | |
| ▲ | tcfhgj 2 days ago | parent [-] | | planned obsolescence by Google | | |
| ▲ | shadowgovt 2 days ago | parent [-] | | Beginning to use a CPU opcode that is 19 years old doesn't feel like planned obsolescence. if anything, it feels like unplanned obsolescence... "Oh hell what do you mean your CPU doesn't have that opcode no we've just been running the compiler with the default flags and that opcode got added to the default two months ago after a 10-year fight about the possible consequences of changing defaults!" Although I'm a little surprised to learn that the binary itself doesn't have enough information in its header to be able to declare that it needs SSSE3 to be executed; that feels like something that should be statically-analyzed-and-cached to avoid a lot of debugging headaches. | | |
| ▲ | tcfhgj 2 days ago | parent [-] | | > "Oh hell what do you mean your CPU doesn't have that opcode [...]" hobbyst dev? sure Google? nope | | |
| ▲ | shadowgovt 2 days ago | parent [-] | | Did they make any explicit guarantees that their newly-cut binaries would continue to support 20-year-old architectures? Googlers aren't gods. It's a 100,000-person company; they're as vulnerable to "We didn't really think of that one way or the other" as anyone else. ETA: It's actually not even Google code that changed (directly); Gradle apparently began requiring SSSE3 (https://gitlab.com/fdroid/admin/-/issues/593#note_2681207153) and Google's toolchain just consumed the new constraint from its upstream. Here, I'm not surprised at all; Google is not the kind of firm that keeps a test-lab of older hardware for every application they ship, so (particularly for their dev tooling) "It worked on my machine" is probably ship-worthy. I bet they don't even have an explicit architecture target for the Android build toolchain beyond the company's default (which is generally "The two most recent versions" of whatever we're talking about). |
|
|
|
| |
| ▲ | Angius 2 days ago | parent | prev [-] | | They clearly don't |
| |
| ▲ | lupusreal 3 days ago | parent | prev | next [-] | | Probably a case of "don't fix it if it ain't broke" keeping old machines in service too long, so now they broke. | | | |
| ▲ | Perz1val 3 days ago | parent | prev [-] | | Yeah and everybody was complaining how slow the builds are for years. I really want to know too |
|
