Remix.run Logo
phpnode 14 hours ago

Not a panacea for broken culture.

I've just left a company that used to have an internal RFC process and it was a very significant barrier to progress that stifled innovation, led to breakdown of working relationships and caused the most productive engineers to run for the exits.

RFC is a request for comments, and it turns out you have to be really careful about the kinds of comments you solicit and who from. As soon as you ask people to comment you are setting an expectation that you will take their feedback onboard and address their points, but there’s a real asymmetry here - it is much easier to leave a critical comment, or to ask a question, than it is to address the concerns or answer the question.

This asymmetrical nature puts much more work on the shoulders of the RFC author. Similarly, the people writing the RFC have typically thought much harder and longer and deeper about the problem than the people giving feedback on it. This leads to authors having to re-explain their thinking in detail, covering points that they’d omitted for brevity or because they are obvious to those with a good understanding of the problem.

Suddenly, the task changes from shipping the feature or making the change to achieving consensus on the RFC. I have seen that process take months - far longer, and being far more expensive than just doing the work would be.

The worst part is - no one is in the wrong! The authors write the RFC in good faith, the commenters review the RFC in good faith, and they rightly expect that any problems they identify are addressed. But the whole process can be soul crushingly slow, painful and full of professional conflict.

I’m not saying that RFCs are bad overall, but you have to have a culture of accountability, pragmatism and getting things done to actually make this process work.

mohsen1 13 hours ago | parent | next [-]

Folks with big titles will always write comments that sound smart and thoughtful but in reality hinder the process. For example:

- This architecture binds us to AWS. Have we estimated the engineering effort to remain cloud-agnostic in case we need to move to Azure next year?

- I see we're using Postgres. Have we considered how we’ll handle horizontal sharding if our user base grows by 1000x in Q4?

- This synchronous API call introduces tight coupling. Shouldn't this be an event-driven architecture to handle back-pressure?

All sound like things that are easy to ask, sound prudent to management, but are impossibly expensive to answer or implement for a feature that just needs to ship.

zamadatix 12 hours ago | parent | next [-]

They only hinder the process if you treat them as demands instead of questions or comments. The questions are generally smart and thoughtful, just often mistimed/misplaced for where most companies decide to have the RFC process (i.e. often after completion rather than before starting main implementation).

It's alright to answer "No, we haven't done estimations for cloud-agnostic architecture as part of this project since it was not part of the approved goals and requirements. If we decide to go multi-cloud at some point in the future, this architecture will need to be reviewed with the rest of the infrastructure as part of the migration plan".

If that kind of answer creates a problem then the issue wasn't really to do with the RFC or the comments, that's just where the other issues in the process became apparent. Namely, the requirements are being set without all relevant stakeholders involved or even aware (which is not the same thing as agreeing).

JamesSwift 11 hours ago | parent [-]

Agree 100%. I dont see how this is any different than PR review. There is nothing wrong with "no, thats out of scope for now". If the argument is that this answer causes problems, then yes thats an organizational issue. But fundamentally, those kinds of questions are exactly what you should hope to get out of this process by looping in others. A higher level and greater diversity of perspectives.

foobarian 9 hours ago | parent | next [-]

I find that it's common for small groups to find themselves in a self-reinforcing purity spiral where they are afraid to say "No" to such comments, and they have no role model to get them out of the bad local minimum.

phpnode 11 hours ago | parent | prev [-]

The main difference with PR review is that code is tangible and real and carries more weight than a document full of plans and ideas. There is a broader acknowledgement of the cost of changing working code, but that price-sensitivity seems to evaporate when it comes to RFCs.

JamesSwift 9 hours ago | parent | next [-]

Again, I think you are assigning too much importance to comments/questions in the RFC. Yes, there is probably an expectation that a comment/question is _acknowledged_, but I disagree there should be an expectation that it is _resolved_. The same as a PR.

If in a PR I left the comment that 'This architecture binds us to AWS. Have we estimated the engineering effort to remain cloud-agnostic in case we need to move to Azure next year?", it would be bad form for you to ignore the question and merge. It would be totally acceptable to either say "no, I didnt take that into account and I think its out of scope" or "yes, and that will be tackled separately". And it should always come with a consideration that there might need to be more information added to the PR, eg adding clarifying comments to the code.

bccdee 7 hours ago | parent | prev [-]

If you're making a proposal, I feel like an RFC carries a lot more weight than a slideshow or an email. If you really don't need feedback or approval from anyone, sure, go it alone. But if you do need or want to run it past someone, let the document you send them reflect the amount of thought you've put into it, and then maybe they'll hesitate before going off half-cocked & suggesting some idea you already considered.

jjmarr 12 hours ago | parent | prev | next [-]

The folks with big titles need to determine if the company's technical strategy is cloud-agnostic and whether 1000x growth in Q4 is a legitimate concern.

If Big Title wants to own the schedule impact they can make these demands.

Maybe I'm not read into the secret deal with Microsoft for next quarter that'll require all 3 of these.

cbsks 12 hours ago | parent | prev | next [-]

I would open a new bug for each of those questions and say “we will evaluate this after the MVP is implemented”. Give the person credit in the bug description. That will usually satisfy their concerns. Set the priority on the bugs to low and I’ll never even have to look at them again, unless one of them actually becomes a problem.

9 hours ago | parent | prev | next [-]
[deleted]
11 hours ago | parent | prev | next [-]
[deleted]
herpdyderp 12 hours ago | parent | prev | next [-]

My favorite response to the unexpected "have we considered" questions are "no". I'm telling the truth, and often there is no follow up!

esafak 10 hours ago | parent [-]

That does not reflect well on you, and these people may be on your promotion committee.

jkubicek 9 hours ago | parent [-]

Maybe, but also having those hard conversations and delivering a well-scoped project on time will probably overcome any minor butt-hurt from being told "no".

zamadatix 6 hours ago | parent | next [-]

Most people don't mind the being told "no", they just want to understand why and how that might affect them as part of that. For anything but very simple questions (like "do I need to change anything about the way we do CI for this" -> "Nope!") you'll want more than one word to accomplish that communication.

I think when most people reference saying "no" they are really referencing saying "no, %{backgroundOfWhy}" or similar often in a few short sentences, and that's great for everyone all around. It's just the literal "no" with no effort to engage or care about why they are asking that will leave a black mark of "Joe Schmoe is really great at delivering... when he feels like talking to you about things".

mohsen1 9 hours ago | parent | prev [-]

that's not how humans work

buescher 12 hours ago | parent | prev | next [-]

They're looking for the level of answer you might get instantly from an llm. I think figuring out the right balance of "ask chatgpt", critical thought, and ownership in these situations is going to be tricky.

Groxx 12 hours ago | parent | prev [-]

In part because of the drive-by questions, they also often force waterfall-like design: few like the answer to their question being "idk, we'll find out when we get there".

So rather than a lightweight doc that looks for obvious gaps, it becomes a giant playbook for every eventuality. While the company claims they're being "agile".

zamadatix 12 hours ago | parent | prev | next [-]

You don't really want company wide consensus on every point in the RFC. That should be reserved between the author(s) and their immediate teams. Setting/allowing the expectation that any comment made will be reviewed and responded to until everyone is in agreement about it is going to make things miserable for everyone and improve nothing.

I.e. that culture of pragmatism needs to be defined as part of the expectations process, not an unspoken promise between coworkers. No process is perfect but a footgun process is easy to make anyways.

Wilder7977 13 hours ago | parent | prev | next [-]

In my organization we have RFCs, PRDs, ADRs etc. and I would say that the process is fairly broken. That said, I think what you mention is an important but not the only failure mode of a proposals process.

In some cases I have seen, people use RFCs to steamroll decisions when they are the only stakeholders. Here the waste comes from the fact that the proposal becomes just a bureaucratic step or a way to diffuse responsibility.

In the case you mention (which I have seen many times) I would say the general issue is that the goals and the constraints are not qualified sufficiently. If that's the case, then there are only 2 cases: there is an objective way to measure if an objection or comment makes sense or not, or it is subjective. If it's objective, then consensus is easy to reach, if it's subjective, it needs to be acknowledged and usually the decisions falls on those who are responsible for the outcome (e.g., the team who needs to maintain or build the thing).

Of course, the debate can move to constraints, goals or even ways to measure, but these are generally more straightforward.

nwmcsween 6 hours ago | parent [-]

Have you worked at an organization without RFCs, ADRs, etc? The alternative is really just the wild west and whatever politics or pull a person has. RFCs and ADRs are good in the sense that they document _something_ even if the document is junk it's better than an assumption.

Really though it's the organization (and people) that makes or breaks anything.

bccdee 7 hours ago | parent | prev | next [-]

> This leads to authors having to re-explain their thinking in detail

What I like about RFCs (or similar documents) is the ways they work to prevent this. Recently I was involved in planning an initiative without a document like this, and we had to keep explaining and re-explaining the motivation for our decisions to stakeholders and higher-ups. With a document (assuming everyone reads the document before giving feedback), most questions get pre-empted; the ones that don't only need to be addressed once, because the answers end up in the version of the doc which you show to the next person.

Certainly I think it's worth being selective about who you're soliciting comments from, to avoid a too-many-cooks situation, but rare is the project that doesn't need anyone's approval or feedback. Presenting a big fat document gives a sense for the amount of thought that has gone into the design, which quells the kind of off-the-cuff "why not X?" comment you might get in response to a boxes-and-arrows chart and a high-level summary.

baby 13 hours ago | parent | prev | next [-]

There is no perfect process. I think after a long time working in software I now understand that it works like this: you need one person to work by themselves really really quickly to ship and create the foundations. After that, they need to remain in place to make sure the design continues to make sense as more people add code to the monolith. Having a specification process is a must in complex environments, and the downside is that it adds friction (and some people can abuse the back and forth to prevent you from landing your design changes), so you need that process to happen as late as possible. But if you're running in production in sensible environments then you will need to make that process happen earlier than desired.

phpnode 11 hours ago | parent [-]

> you need one person to work by themselves really really quickly to ship and create the foundations

This is literally the secret of every successful project I've worked on. An individual, with high agency and low communication overhead has either been trusted to take the task on themselves, or they've done it secretly to side-step the approval process and organisational overhead. When the foundations are in place it's relatively easy to add engineers, but starting something from scratch with a bunch of people? doomed before it starts in most cases imo

uberduper 13 hours ago | parent | prev | next [-]

My take is that an RFC should be very early in the engineering process, like as part of a proof of concept phase, and should not block progress towards completing a design proposal. The design proposal should list any legitimate alternatives to overall or component designs discussed during RFC along with the reasoning for not using them in a "designs not chosen" appendix. This at least gives your engineering leadership an opportunity to evaluate the general design ideas before anyone is prepared to die on the hill of those ideas.

Architecture / Design review happens post proof of concept but still before any significant development work and major action items are blockers to beginning development. Further discussion about designs not chosen can happen here, especially when a flaw is uncovered that would be addressed by one of those not chosen.

Phelinofist 13 hours ago | parent | prev | next [-]

> This leads to authors having to re-explain their thinking in detail, covering points that they’d omitted for brevity or because they are obvious to those with a good understanding of the problem.

IMHO there should be no gaps like that. If you already thought hard and long about these problems, why not just write all your knowledge down, so other people can benefit from that? Also, this makes your work a lot more accessible to those who came after you. Of course, you have to assume a certain baseline of knowledge. But if you already have a good understanding of the problem, why not formulate that as part of the RFC?

phpnode 13 hours ago | parent [-]

It is impossible for people to write all their knowledge down, there will always be gaps, there will always be things taken for granted.

The point is that the author thought that they had captured enough information in their original draft - they can expand on that draft of course, but that's the issue - the task of writing the RFC can become much larger than just trying something out and getting real data.

elktown 13 hours ago | parent | prev | next [-]

Like many things in dev it sounds sooo good on the surface, but is a minefield in practice (Brandolini's law + The Iron Law of Bureaucracy for starters).

I'd only advocate it in a very carefully curated team.

uberduper 13 hours ago | parent | prev | next [-]

> This leads to authors having to re-explain their thinking in detail, covering points that they’d omitted for brevity or because they are obvious to those with a good understanding of the problem.

There's nothing wrong with this. Being able to explain your thinking in detail to someone that doesn't necessarily understand the problem is a pretty good exercise to make sure you yourself fully understand the problem _and your thinking._ Of course, this can't turn in to a lecture on basic things people should know or have looked up before commenting.

menaerus 10 hours ago | parent [-]

Sure, now imagine answering 10 different people to all of their questions? It's the largest hindrance I have ever seen but I agree with the above comment that it largely depends on the team.

danielvaughn 12 hours ago | parent | prev | next [-]

Yeah I think it all boils down to culture. Tools like RFC (and anything else) can help propel a good culture forward. But you can't fix a broken culture with a tool.

kscarlet 13 hours ago | parent | prev | next [-]

I find it funny that what you said summarize the publishing process in academia very well. Only except that it's much, much worse.

a012 12 hours ago | parent | prev | next [-]

This is my experience as well, the RFCs turn out are being made for internal processes and policy making. And also I have the feeling that people are making it like Request for Approval for technical changes, so we have to take significantly more time on the RFCs than the real work.

throwaway894345 13 hours ago | parent | prev | next [-]

I feel this tension, but I think something has gone awry if people feel like every comment has to be addressed to everyone’s satisfaction. Comments are not commitments, and commenters don’t have an equal stake in the decision—they certainly don’t own the decision, and it’s okay to disagree and commit.

It’s also probably worth being explicit that there is a cost to inaction that can exceed the costs of building the wrong thing. A year or two ago I was lead on a project where we didn’t know the answer to a big ambiguous problem—we just didn’t have a good way to get the information necessary to make the right decision—different people had different ideas about what we should do. So we identified the smallest thing we could build that would be useful such that we could get more information from real world use, knowing full well we might have to rebuild something else from the ground up. And we did! But we were able to get the confidence we needed to build the right thing later! And we got there much faster than if we had tried to deliberate and speculate about what that thing would be.

Also, in that saga, one engineer in particular was really adamant that we address his particular set of concerns. He was unable to disagree and commit—he was kind of religious about all concerns being addressed before moving forward with anything. He had a very difficult time understanding that the RFC process existed to meet business goals—the business did not exist to slot into his ideal RFC process. He is no longer with the company.

12 hours ago | parent | prev [-]
[deleted]