Remix.run Logo
gtowey 8 hours ago

Yup. You learn the most valuable information from watching how things break and then fixing them.

It's kind of like when the FAA does crash investigation -- a stunning amount of engineering and process insights have been generated by such work to the benefit of all of us.

crystal_revenge 7 hours ago | parent | next [-]

> You learn the most valuable information from watching how things break and then fixing them.

Trust me, you get plenty of experience in this as a founding engineer in a startup.

Many of these comments make me wonder how many people here have actually worked at an early stage startup in a lead role. You learn a lot about what's maintainable and scalable, what breaks and what doesn't, in the process rapidly iterating on a product to find your market.

Karrot_Kream 5 hours ago | parent | next [-]

I don't think HN has been frequented by startup engineers with leadership responsibilities in any density in a long time. It's very obvious to me reading a lot of the comments here that most folks are ICs somewhere in a large, bureaucratic software organization. That's why there's so much BOFH style commentary here these days.

(For readers, I don't think there's anything wrong with that but it just means that certain perspectives are overrepresented here that may not be more reflective of the broader industry.)

crystal_revenge 5 hours ago | parent [-]

That's what I've come to realize. For most of the commentators here "greenfield" means typing in 'npm init', for me it usually means doing three different roles in order to iterate as fast as possible on the product to find your market, then figuring out how to scale it to the new users you've started acquiring.

The idea that this is means "you don’t learn the actually valuable lessons" is completely baffling to me.

Most people I've know with founding engineer experience or similar leave not because it's not challenging, but because it's exhausting.

Increasingly I've realized that the HN community and I are not even speaking the same language.

Karrot_Kream 5 hours ago | parent | next [-]

Some of the sharpest engineers I knew built tools and business processes at startups and watched them fail as they scale. I ran an internal presentation for years at a Unicorn where I was an early employee called "Failure at Scale" where I tried to capture lessons of huge incidents we had that were caused by us crossing scaling thresholds. Eventually the presentations stopped being meaningful because the company became too big and too removed from its origins.

edgyquant 4 hours ago | parent | prev [-]

No but it usually doesn’t mean achieving stability at tens of thousands of users a day (or hour) and ensuring that stability while rolling out new features, migrating infrastructure etc

Karrot_Kream 2 hours ago | parent [-]

You definitely do. Do you think Anthropic isn't working with thousands of users an hour? They're struggling to keep up with the scale and their ability to create a stable platform is, well, existential for them. Do you think Anthropic isn't a startup? The pace of their feature rollouts is exponential.

Even in areas where startups aren't literally creating new product categories like the foundational model providers, the edge of a startup over a more established business is the speed at which they can provide value. What's the point of buying CoolCo when you can go with L&M Inc. that has thousands of headcount working on your feature. The value prop of CoolCo is that CoolCo can roll out a feature in the time it takes L&M to make a detailed specification and a quarterly planning doc breaking down the roadmap and the order of feature implementation.

esseph 5 hours ago | parent | prev | next [-]

> Trust me, you get plenty of experience in this as a founding engineer in a startup.

Now be part of the team of folks that keeps that application running for 10, 20, 30 years. Now be part of the transition team to the new app with the old data. Those tasks will also teach you a lot about system stability, longevity, and portability... lessons that can only be learned with more time than a startup has.

cyberax 41 minutes ago | parent | prev [-]

I'm a founding engineer in a startup right now, and I was a founding engineer in a startup acquired by a large company. So then I became a part of a large company.

The technical challenges are _very_ different between these environments. In a small company you have to deal with technical breakages all the time, but you don't really have systems-level problems.

usrusr 3 hours ago | parent | prev | next [-]

Valuable in what metric? I'm very much in the brownfield-has-the-lessons camp, but one of the lessons is that this experience has a very low market value. In fact it's so impossible to downgrade from "senior in $outdated" to "junior in $whateverisconsideredhotrightnow" that any brownfield experience could easily be considered to have negative market value.

afavour 7 hours ago | parent | prev | next [-]

I don't think they're mutually exclusive. You could just as easily describe someone with bootstrapping experience as being like an FAA crash investigator who investigates take offs. You get to know exactly what works when moving fast and looking for quick results, and what dooms a short timeline to failure.

aeonfox 6 hours ago | parent | next [-]

> You could just as easily describe someone with bootstrapping experience as being like an FAA crash investigator who investigates take offs.

Takeoff systems aren't analogous to prototype development. I don't know you'd build a prototype plane that's feasible to take to market, without having deep knowledge about how planes are built.

Early design decisions matter. And you don't get to that realisation without dealing with legacy systems where some upstart made terrible decisions that you're now responsible for.

5 hours ago | parent | prev [-]
[deleted]
brightball 2 hours ago | parent | prev | next [-]

Yep. I have loved fixing problems in systems and the processes that the people working on them use for years.

detourdog 7 hours ago | parent | prev | next [-]

I was just thinking yesterday that knowing all the ways something breaks and behaves is the key to understanding systems.

Vibeguy900 5 hours ago | parent [-]

and people that understand data it critical with storage supply getting tighter

toomuchtodo 8 hours ago | parent | prev [-]

“What kind of role are you looking for?”

“Technologist flavor of NTSB investigator.”

eitally 7 hours ago | parent | next [-]

There was a two year period around 2011-2013 where I experimented with my dev team. We were being "forced" to migrate a legacy enterprise system from .Net/MSSQL to Java/PostgreSQL with replace the front end with a modern, reactive web interface. Only two of the existing developers had Java experience, and both were conveniently senior engineers in two different offices and running their own discrete sub-teams.

One of the guys had a very strong opinion that the ideal architecture was something as abstracted and object oriented as possible with single function classes, etc. I let him run with that. The other guy got frustrated with his sub-team's inability to write code to spec in a language they'd never used before and where they were trying to build some new features they didn't clearly understand. He developed a strong feeling that TDD was the most efficient path forward: he owns the PRD and design, so he just created test stubs and told the remote team to "just write code that passes the test" even if they didn't understand the function of the block.

So, after a few months where did we end up:

1. The "abstract everything" architect's team had an extremely fragile and impossible to maintain codebase because it was impossible for any outsider to tell what was going on.

2. The "just pass the damn tests" guy had a team that had quickly ramped on a new language and they had a codebase that was incomplete (because they were building it like a Lego project) but that everyone could understand because the code blocks generally stood on their own.

What was the next step: to shut down the guy who abstracted everything and force him to drive a quick & dirty rewrite that would be more maintainable, and to also start a major refactoring of the "Lego" team's code because it was so fragmented that it also was fragile and unsuited for production.

I saw this as a terrific learning experience for all involved and I was able to get away with it because the stakes were pretty low and we had time to experiment (part of the ultimate objective was upskilling the team), but the more important lessons were these:

1. Docs matter. Take the time to write clear & detailed specs first because you'll be forced to think of edge cases and functionality that you didn't originally, and it provides a basis for system design, too.

2. Architecture & design matter. Adhering too close to any single paradigm is probably a mistake, but it takes experience on the team to understand where the compromises are and make the best decision for that system.

That second point will not stop being true with the advent of agentic assisted software development. Like others have said, my expectation in the job market is that pay will continue to be depressed for junior hires as employers reset expectations and generally just want folks who can instruct fleets of agents to do the actual coding. Senior staff will become increasingly critical and their jobs will be painful and difficult, because it'll be assumed they can (and will be willing to) do design & code reviews of artifacts originated by agents.

What I am going to be most interested in is what happens in the SRE/Sysadmin world over the next few years as more AI-generated code hits prod in organizations that don't have adequate review & oversight functions.

jordanb 6 hours ago | parent | next [-]

The nice thing about the "just pass the tests" approach is you end up with something sufficiently abstracted that it's easy to write tests for it.

__turbobrew__ 3 hours ago | parent | prev [-]

> What I am going to be most interested in is what happens in the SRE/Sysadmin world over the next few years as more AI-generated code hits prod in organizations that don't have adequate review & oversight functions.

You kindof answered the question yourself. Humans write the tests and then go tell the AI to write the solution which passes the test.

hn_go_brrrrr 8 hours ago | parent | prev [-]

Sounds amazing.