| ▲ | jotux 3 hours ago |
| Cool coincidence. In a past job, I specifically made replacement PCMCIA cards for these (and other) planes. Instead of making cards with a controller and onboard storage, we made cards with an FPGA/microcontroller and a microSD card soldered to the PCB. We did this specifically because these aircraft use (at least partially) broken implementations of the PCMCIA protocol, and we'd update the bus protocol per aircraft to work around the limitations. None of these planes can just go buy stuff off the street because some large portion just doesn't work because their protocol isn't fully implemented and isn't flexible to deal with variance in timing/speed/etc. A fun story: Our cards had whatever industrial-spec microSD card we could find (16GB, 32GB, etc.), and we'd internally limit the disk size to whatever the platform needed. Almost every aircraft had some specific number of MB they wanted because all of their CONOPS listed it, and they didn't want to update documentation or software to put a different number there. We had a plane that could only use EXACTLY 128MB, but they were currently using 256MB cards and partitioning them to 128MB. We updated the disk size to present as 128MB so they could skip this step, and they specifically told us to put it back to 256MB because they didn't want to update the commissioning process to remove the partitioning step. It was rewarding work, and the card we designed was cheap (~$500 each, which is literally nothing for major platforms like this) and 100% US-designed and made (with the exception of ICs manufactured in the usual places). Still, the procedural and cultural practices of these program offices made it a real struggle to get traction and deploy them widely. Government acquisition is just generally designed around funding primes for current development and then paying through the nose for sustainment later. It was hard to design something for widespread sustainment use because by the time you were done and ready for production, a bunch of parts would be obsolete. Then you'd have to rev the design over and over. We were never allowed to stockpile anything, even if it was trivially cheap, to facilitate production later. |
|
| ▲ | lwejrlkj 9 minutes ago | parent | next [-] |
| I worked on submarines in the 2000s and there was a whole cottage industry of old men who maintained the weapons control systems. The Navy wasn't allowed to upgrade the weapons control computers, as far as I understood, in part because of nuclear weapons limitation treaties, so they had to keep the dinosaurs running. 14in HDDs and analog circuitry and all the rest. And all of the companies that built them, all of the manufacturing lines and tooling and processes, were long gone. |
|
| ▲ | criddell 2 hours ago | parent | prev | next [-] |
| > We were never allowed to stockpile anything Now allowed by whom? Was it a business decision or a regulation? |
| |
| ▲ | jotux 2 hours ago | parent | next [-] | | Without going into a ton of detail, I was working as a federal employee for a federal laboratory. Per regulation we couldn't purchase anything not directly tied to an agreement, and we weren't allowed to complete with industry. Typical work would look like this: A group would come to us and say, "We use these cards, but they wear out and we need like 200 more to keep running for 15 more years, we talked to primes about fixing this problem for us but they said the only thing they're willing to do is redesign the entire data handling infrastructure in the plane for $500M." So we'd get some tiny amount of money (like $200k) and redesign the card and make them ~20 prototypes. It'd work great, then they'd spend 3 years working on production contracts to try to get 200 built for $100k. In 3 years, some part would go obsolete, they'd come back and give us $50k to update the design and test, then start the 3 years cycle to buy 200 again. Rinse and repeat. If they would have just given us like $30k, we already knew which parts were high risk of going obsolete, we could have just bought everything necessary to build 1000 units and dropped them in storage, but federal regulation limited what was possible. SUPER frustrating to have something that solves a problem and watch it fall apart over crappy policy. It's sad to see this request, knowing that I was part of the team that designed a solution specifically to solve it, knowing it worked and tested it on the platform, but some part probably went obsolete and they never got enough manufactured to solve the problem permanently. | | |
| ▲ | dan_p 2 hours ago | parent | next [-] | | This is a microcosm of much of what's wrong with government. Interesting story, thanks for sharing. | |
| ▲ | criddell 2 hours ago | parent | prev [-] | | Sounds like an opportunity. Form "Jotux Electronic Supply, LLC" (JES) and stockpile the stuff that's cheap now but you believe is gong to be unavailable in the future. When they come back in a few years, the parts are available from JES, they are just much more expensive now. | | |
| ▲ | jotux 2 hours ago | parent [-] | | This was a regular and consistent joke we made, and we saw contractors do specifically this as well. All of our work was intended to be advertised and eventually produced by multiple manufacturing contractors. One contractor would see that a specific part was going obsolete and go buy all of them, and basically starve out the competition. After they were the only contractor that could actually build it the price they would quote would go up 10x. |
|
| |
| ▲ | lwejrlkj 8 minutes ago | parent | prev | next [-] | | If Congress didn't write spares into the contract twenty years ago, then you don't have spares | |
| ▲ | 2 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | jacquesm 2 hours ago | parent | prev [-] |
| This sounds so wasteful. |
| |
| ▲ | russdill 13 minutes ago | parent | next [-] | | When you're dealing with smaller quantities and high engineering costs, the math changes drastically. Wasteful here is relative. Wasteful relative to what? Certainly not wasteful relative to upgrading all the existing hardware to take microsd cards | |
| ▲ | jotux 2 hours ago | parent | prev | next [-] | | That's because it is. The FAR is trash and should be thrown out and re-written, but congress isn't operational enough to do it and primes have plowed too many regulations into it to ever let them give it up. | | |
| ▲ | runamuck 44 minutes ago | parent | next [-] | | In my experience, trying to get the government to adopt innovation feels cumbersome and "paperwork-first." I've brought disruptive tech to some FedCiv orgs and they would say things like "we don't have a line item for that in our budget." Luckily, things like OTAs signal an attempt on their part to make the process better. (I'd recommend the book Unit X by Raj M. Shah for a history behind the push to improve the acquisition process). | |
| ▲ | Lt_Riza_Hawkeye an hour ago | parent | prev [-] | | are you referring to the the Federal Aeronautics Regulations aka 14 CFR? | | |
| |
| ▲ | dfxm12 an hour ago | parent | prev [-] | | they didn't want to update the commissioning process to remove the partitioning step Depending on regulations, updating the processes and associated documentation might have been more wasteful with regards to time to update, review and update the processes, time to get everyone retrained, risks of being out of compliance due to confusion, materials to print everything out, etc. | | |
| ▲ | jotux 40 minutes ago | parent [-] | | Yes, 100% agree. Another common problem we saw was that most major systems were owned by one specific contractor. Lockheed owns f-22, Boeing owns f-15, etc. Oftentimes there was some requirement to use the process as defined and delivered from the contractor. Want to update a document to remove step 7 from the data loading process? Welp, you'll need to get them on contract to update the document, verify it, regenerated documentation, train users to stop doing step 7, and reissue to the PO. That will only be $1.5M. I'm not saying it's bad that these processes are in place, they make sense for systems this large, expensive, complicated and important. But there once was a time where government employees owned a bunch of this and when there was a problem they could just go fix it. That has been eroded, and prime contractor dependency has been enshrined in the way we purchase and maintain these systems with the sole intent of giving more money to those contractors. Pure regulatory capture: https://en.wikipedia.org/wiki/Regulatory_capture |
|
|