| ▲ | YZF 2 days ago |
| I've been in the industry for >30 years ;) I'm not sure what's the proposal? - Don't use an OS. Write your own. Linux? Boring. - Design your own CPU. - ext3 or xfs? Nah write your own. - Write your own database. - Ethernet. Too boring. Create a new physical layer for networking. - Naturally create your own programming language. That'll make it much easier to find people when you have to expand the team. Seriously, build vs. buy and NIH has always been with us. There's a time to build and there's a time to buy/reuse. Where are you adding value? What's good enough for your project? What's strategic vs. tactical? How easy is it to change your mind down the road? What are the technical capabilities of the team? How do different options impact your schedule/costs? How do they impact quality? In the short term? In the medium term? In the long term. |
|
| ▲ | ATMLOTTOBEER 2 days ago | parent | next [-] |
| I’ve been in software for over 40 years (yes I’m that old ), and in my humble opinion it’s always correct to build. It keeps things fresh. |
| |
| ▲ | YZF 2 days ago | parent [-] | | The reality is there is no way to build everything. You want to do scientific computing do you use libraries that have been optimized for 50 years or do you write your own? You want to do cryptography do you build your own? Pretty much everyone working on LLMs today is leveraging things like nccl, cuda, pyTorch, job scheduling frameworks. Let's face it. Nobody builds everything from scratch. The closest is companies like Google who due to sheer scale benefit from building everything from hardware to languages and even for them it's not always clear whether that was the right thing for the business or something they could afford to do because they had lots of money. Build the things that add value. Don't build something that just works. That's why we have the old saying don't reinvent the wheel. If you have a working wheel, while re-inventing it might be fun, it's usually not the best use of time. In the time you've saved build cool things that add value. | | |
| ▲ | tehjoker 2 days ago | parent [-] | | gotta say, having written some scientific computing code, the libraries out there do not always cover the exact operation you need and are not always using the best algo in the literature. i was able to beat the existing ecosystem 6x head to head and thousands of times faster for my use case. ymmv ofc depending on the problem. that said, it was not easy!! | | |
| ▲ | YZF a day ago | parent [-] | | I worked on some proprietary video/image encoding application. In that context we hand wrote things like colour space conversions, wavelet transforms, arithmetic coders, compression algorithms, even hashing functions, in SIMD and we got better performance than anything off the shelf. We still used some off the shelf code where we could (e.g. Intel's hand written libraries). The thing is that this was the core of our business and our success depended on how performant these pieces were. That was also some time back, maybe today the situation is different. In this sort of situation you should absolutely put in the effort. But that typically accounts for some small % of the overall software you're going to be a user of. This is really just another variation of the premature optimization statement: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%". So if you're in the 3% then by all means go for it (you gotta). But if you're in the 97% it's silly to not use what's out there (other than for fun, learning etc.) |
|
|
|
|
| ▲ | spookie 2 days ago | parent | prev | next [-] |
| Let's be honest, nobody is saying to rebuild the world from scratch. The stance for in-house built tools and software is a much more balanced act than that. One that prioritises self-reliance, and foments institutional knowledge while assessing the risks of making that one more thing in-house. It promotes a culture where employees stay, because they know they might be able to create great impact. It also has the potential to cut down the fat of a lot of money being spent on third parties. Let's be real, most companies have built Empire State Buildings out of cards. Their devs spend most of their time fixing obtuse problems they don't understand, and I'm not talking about programming, but in their build processes and dependency hell. It's no wonder that the giants of today, who have survived multiple crisis, are the ones who took the risk of listening to those "novice" enthusiastic engineers. Don't kill the enthusiasm, tame it. |
| |
| ▲ | YZF 2 days ago | parent [-] | | Sure. We should harness enthusiasm and channel it in the right direction. I'm not sure I agree the giants of today are built on the work of enthusiastic novices. Amazon and Microsoft have always had a ton of senior talent. Meta started with novices but then a lot got reworked by more experienced people. You might get by with sheer enthusiasm and no experience but often that leads you to failure. | | |
|
|
| ▲ | ConspiracyFact 2 days ago | parent | prev | next [-] |
| This is a bit disingenuous, don't you think? There's technically a spectrum from low-level to high-level code, but in practice it's not too difficult to set a limit on how far down the stack you're willing to go while building. Writing a new testing framework is qualitatively different from writing a new OS or filesystem, and you know it just as well as everyone else does. |
| |
| ▲ | YZF 2 days ago | parent [-] | | I was trying to make a point... Apparently not very well ;) But let's take on the testing framework question. When I work in Go or in Python I use the testing frameworks that are part of that ecosystem. When I worked with C++ I'd use the Boost testing framework. Or Google's open source testing framework. Engineers I work with that do front-end development use Playwright (and I'm sure some other existing framework for unit tests). Can't you do your own thing? sure. You'd be solving a lot of problems that other people have already solved. Over time the thing you did on your own will require more work. You need to weigh all of that vs. using something off the shelf. 9 times out of 10, most people, should use tooling that exists and focus on the actual product they want to build. That said I work for a large company where we build a lot of additional in-house tooling. It makes sense because: a) it's a specialized domain - there's nothing off the shelf that addresses our specific needs. b) we are very large and so building custom tools is an investment that can be justified over the entire engineering team. We still use a ton of stuff off the shelf. I don't see what the parent was saying. I think most of the time people choose to use existing bits for good reasons. When I started my career you pretty much had to do most stuff yourself. These days you almost always can find a good off the shelf option for most "standard" things you're trying to do. If you want to write your own testing framework for fun, go for it. If you're trying to get something else done (for business or other reasons) it's not something that usually makes sense. That said it's not like we have a shortage of people trying to do new things or revisit old things, for fun or profit. We have more than ever (simply because we have more than ever people doing software in general). |
|
|
| ▲ | yellowapple 2 days ago | parent | prev [-] |
| "If you wish to make an apple pie from scratch, you must first invent the universe." |
