I've worked at two companies with RFC processes. At both companies, the processes were -- and I hate to use such a strong word -- a total sham.
I suppose the processes were crafted to look like some egalitarian meritocracy. In reality, it was just a test of who had the most political pull. If you had management on your side, your "RFC" was effectively law and dissenting voices were effectively career suicide.
Which, you know... fine. I understand that choices are made based on cliques and political capital rather than anything else. Cool. That's okay! Sucks sometimes, but that's how the world works. Hopefully your org structure itself is at least something of a meritocracy, so that edicts from on high are of a generally high-enough quality. If they get it right-ish often enough, your org will be okay... probably.
But what really rankled me was the fact that the RFC processes amounted to some kind of elaborate cosplay so that, I guess, folks could pretend that there was some sort of healthy collaborative process. If you're going to let a few "popular and management-blessed" engineers make all the decisions, fine, but don't add insult to injury by pretending otherwise.
I've heard similar things from others, elsewhere. I hope there are at least a few companies dedicated to being something better.
I think the most important part of these processes is to document what’s being proposed and have a period for comments / inputs. I also think it’s ok for a monarchy as opposed to democracy ruling what gets approved, assuming merit based is not an option.
In general merits (rationality?) can bubble up if it’s framed in correct way for upper levels (eg. risk, profit, time horizon), and can then sway the position.
But yes on average what’s decide at upper levels is gospel. But that’s how corporations function, and that’s ok.
> I also think it’s ok for a monarchy as opposed to democracy ruling what gets approved, assuming merit based is not an option.
Just to share: I've been an outside advisor to a few companies where some developers may think "this is a politics based monarchy and we're not looking at proposals rationally". Where as an outsider it was clear to me that the higher up roles just had more context to make their decisions.
Especially decisions coming from the CTO and VP Engineering kind of roles tended to be driven by input from the wider company strategy, due to daily interaction with the rest of the management team.
In those cases there was a clear need for coaching the higher ups on how to explain and communicate their decisions, but what they were deciding were the right things. And everybody leaving before they improved communications would be on here posting about how bad their higher ups were and what a political mess it was. While that wasn't really the issue.
I'm sure that's not always the case and big political dumpster fires exist (I've seen a few), but there are also many cases where it's only an issue with communicating more clearly from the (technical) management side of things.
So the RFC process was a sham because it never reflected the "wider company strategy" because the people with that context withdrew from it totally apart from being kingmakers?
I'm not sure you are making the case for it here. You are just making some other unrelated point.
Yeah that tracks with my experience. Sucks when more junior folks blow up over things they don’t have total context on. You give them as much as you can, but sometimes people decide they want to be a prophet of doom and close their minds.
but there are also many cases where it's only an
issue with communicating more clearly from the
(technical) management side of things.
This is true, although I've been through versions of this process where there was pretty good PR effort from the RFC backers (talks, slide decks) but it was an extension of the charade. A lot of the slides were just absolute hooey, and pointing this out did nothing to slow down the process one bit.
They communicated well, but the actual thinking was bad.
Advisor to the CTO on request of their investors after an acquisition. The big challenge was to merge the two technologies.
And advisor to the CEO in a scale-up where they used to have a quite technical founder as the CEO who left and got replaced by a not so technical outside hire.
I'm also trying to get a few more board advisor kind of roles, but so far everything has been with an emphasis on the tech side of the business.
I also think it’s ok for a monarchy as opposed to
democracy ruling what gets approved, assuming merit
based is not an option.
I completely agree. What I object to is the charade!
In general merits (rationality?) can bubble up if
it’s framed in correct way for upper levels
Theoretically, yes, but it's trivially easy to fool upper levels with one-sided technical opinions. Choose the name of any technology or tech-related practice at random, pay me $20, and I'll write you a one-sided RFC about why our imaginary company should adopt it. I'll make it sound really good. Your boss will love it and if they don't, I promise their boss will.
> Choose the name of any technology or tech-related practice at random, pay me $20, and I'll write you a one-sided RFC about why our imaginary company should adopt it. I'll make it sound really good. Your boss will love it and if they don't, I promise their boss will.
Ultimately, business folks must always trust someone with technical expertise, trust is a necessary component of the system. Competent organizations ensure that someone technical has the relevant executive's ear, there are lots of ways to achieve this and it should be able to catch your one-sided RFC if you're proposing anything important.
At the end of the day though, there's no process-based solution for an organization riddled with incompetence. Business must make judgment calls and if your company is staffed with people who consistently make bad calls process will not save you.
That's the intended purpose and origination of these systems. In a traditional hardware firm, marketing is constantly coming up with ideas but you can't constantly change hardware without delaying the project. So you come up with an engineering change request system. This forces marketing to go through a bit of bureaucratic pain every time they want to add something to the project. The committee acts as a gatekeeper but mostly, that little bit of extra work makes marketing self-select the more important ideas to go through the process.
Yep. Things like architectural review boards, RFC processes, corporate-wide approvals, etc are more about politics than anything else.
Often - not always, but often - the people seeking to sit on these committees are the least likely to be able to actually build software. It is a function of these kinds of boards that will invariably draw those who aren’t so great at coding or architecture.
No collaboration process will remove people from the system. You should never expect it to, if you have problems with social power, that's a culture issue which isn't something you can control directly. You can discuss culture issues as a group and nudge toward changes. Process won't define culture - it might only nudge it, but it's very easy for established culture, especially that on the side of social power, to completely ignore the influences intended by process.
What I read from Squares description in the article is a lot of power uncertainty, which is sadly all too common in the software industry. What comes through repeatedly in the problem description is any uncertainty defaulting to no communication. This problem won't be local to the RFC process, and ideally should be tackled more broadly while also looking to improve it in processes. Communication needs to be rewarded even if the technical content is inaccurate - more people need to speak up more, and with less concern, and receivers need to approach the commentary with a positive attitude to facilitate both not feeling pressure and to encourage ongoing communication. The approvers list is for example doing this, though for a privileged few. What will likely come to light later is that the approvers portion of the process is causing an asymmetry of inclusion, further feeding a lack of communication from other parts of the teams.
An immediate concrete thing I would suggest that builds atop the existing described structure is to invite rotations for folks to join the "council" and review processes regularly but temporarily. This will increase inclusion and dilute exclusivity and prestige. Explicitly open space for participation and solicit input from visiting members, and again it is important to reward communication even if it did not have the desired content.
In my experience processes like these rarely work out as intended and usually add layers of bureaucracy for marginal benefit. It’s usually senior engineers or middle managers looking for “org wide impact” so they can get promoted.
At [current job] it's run by a group of architects that aren't responsible for delivering anything. Which works about as well as you might imagine. Had one legit tell me something like "that's for you to figure out" when I pointed out some implementation challenges.
Of course now that the project is behind there's no architect in sight to help us out.
My current role is an architect position where we are hands on, after I turned down two that sounds like they were like what you described. To me it's a red flag if an architect team is separate from the dev team. At a minimum I want to be embedded in the dev team during projects. (One of the places I walked away from the "opportunity" it turned out the architects rarely even talked directly to the dev team)
I recently joined a group of architects that aren't responsible for delivering anything and you are not totally wrong. But it isn't the architect role to involve themselves in weeds of the implementation - that's your job. They should be helping if the project is falling behind, though. Then again, if the project is falling behind, then that is probably more indicative of a poorly-run product team and/or a bad SDLC process.
Then again, if the project is falling behind,
then that is probably more indicative of a
poorly-run product team and/or a bad SDLC process.
That's a worrisomely glib take on the impact that architecture can have on the project especially since in practice "architecture" includes the choice of language, platform, and tooling for a project.
Architecture choices directly or indirectly affect every line of code, every minute, every meeting, everything. As you said there are plenty of other places where things can go bad. But it's galling to hear an architect say, "Probably not the architecture, must be something else."
a group of architects that aren't responsible for
delivering anything
And here's the root of the problem. The craft of delivering software is, well, delivering.
The code itself is often somewhat trivial. Effective delivery is all about working about quirks and potholes and traps in the toolchain and architecture inefficiencies.
As we all know software moves fast, and once you've divorced yourself from the reality of actually delivering anything at all, your knowledge quickly decays. You know abstract concepts and how to draw the rectangles on the whiteboard, but before very long you don't know how anything at all actually happens and soon you are unqualified to tell anybody else how to do it.
There can be exceptions to this rule, like projects with an extraordinarily stable toolchain and environment. If you've been writing COBOL at a bank for 40 years you can probably step back into an advisory role and benefit the company by getting yourself out of the trenches and guiding others instead.
If architects are responsible for shipping features, they won't have time to be platform-wide architects. If you are shipping product-level features, you are on a scrum team and if you are on a scrum team, your other platform-wide duties are not your priority. At some point, companies realize they need people focused on the full picture, and they end up calling them architects.
It's certainly true: architects' primary focus needs to be that architecture/platform work.
Architecture isn't something you can just do on the side when you've got some free time, while still crushing out feature requests and bugfixes 40-50 hours/week.
But.
Architects can't be so detached from shipping code that they have no idea how things are actually made and shipped.
Otherwise you get situations like aloof architecture astronauts casually dreaming up complex distributed or microservice architectures, which look great on the whiteboard, but entail an enormous shift in thinking, tooling, and processes. As well as lots of overhead in general compared to a monolithic architecture. Sometimes that is the right move, but it is a costly one and not a shift to make lightly.
Had another "architect" 20 years ago dream up what was probably the world's dumbest architecture. He demanded that all application layer code be stripped out. We were going to build an entire interactive website with nothing but XML and XSLT. Where conditional logic was required, we could use conditional XSLT <xslt:if> statements. He tried to have us implement a social networking site with that. Had he tried to ship a single bit of code he would have realized it was insane.
Also recently had architecture astronauts push an entirely new primary language on the team. The old one was deprecated. This change was largely championed by people who had never written a line of code in either language with no thought to the tooling changes, process changes, or the overhead of throwing away 10+ years of company-wide language expertise in Language X or the ramp-up time and bruises that would be required to get good at Language Y.
After all, what did they care? They're architects.
One way to avoid/mitigate this is to have architects build and ship proofs-of-concept and reference implementations using the proposed shiny new architecture bits before the rest of the team is expected use and master them. Learn as many of the speed bumps and potholes as possible. They should continue to do this and work closely with scrum teams as those teams build and ship. (That may sound obvious, but lol @ this industry)
With examples like that, I can't blame you at all for being skeptical. Architects should definitely deliver POCs and/or reference projects for totally new stuff.
Yeah. In those negative examples, clearly the root of the problem was "architects making bad calls" and not necessarily a fundamental argument against the existence of architects.
But, when the role is poorly constructed, it's a nearly guaranteed mess.
One could say the opposite extreme is just as bad of course. Without architects, you have a bunch of unherded cats. That's also true.
Must the "platform" be so different from a "product"? I think a platform team should think of their output as a product, and so should the rest of the company. Have a PM, scrum or whatever, normal engineering titles, all the usual trappings.
And then you conveniently avoid the architect title!
I just hope one day I am good enough at programming to draw some boxes, tell someone else to figure it out, and then blame them when it doesn't work out
It's the job of all engineers involved to utterly understand one another. That goes for junior engineers cranking out unit tests as well as for architecture astronauts.
If someone here is a junior and gets "figure it out" from some strategic level engineer, ask professional for clarification or a team-up until you understand. Involve management if the other engineer is uncooperative or fails to help. It's their job to explain slightly more than it's your job to understand.
If all else fails, find an organization that expects the architecture astronauts to be explainers in chief as well.
I find those prejudgement political games damaging as well to the point that I'm fantasizing about absurd ideas like anonymous version of slack/hn like chat where messages don't have identities to push it slightly towards merit based conversations. Upvote on messages/threads, not user accounts etc. I don't mean anonymous usernames, I mean no identity at all or at most random identities per message, sticky on reply chain (to know the original author answered your question at most).
I worked at one company where (at least within our team of about 40) our process was legitimately good. There was a genuine back and forth discussion and good ideas tended to float to the top. Then we got acquired and things steadily changed. Slowly more and more decisions were made because some high level engineer or manager just wanted it that way. Of course, terrible ideas floated to the top and it lead to disastrous decisions.
It is possible for the system to work, but it’s a fragile equilibrium and easily destroyed.
What made the process work so well? What about the culture made it so there was that back and forth? I'm wanting to build this into my teams and keen to hear what's worked for others
Empathy. Our team already had a great culture of empathy and mutual respect. People just tried to see things from other's POV, or if they couldn't, they'd at least be respectful. To behave differently would have just seemed weird. I think that our manager and the founding members of the team set the tone early, and as the team grew, it just became the norm. When we lost that manager, things declined. The worst part is that when I look back at my own behavior, I lost my empathy. I felt like a rat in a cage and I behaved accordingly.
If you already have that kind of empathic culture, then I think it's just about starting a "good enough" process and refining it as you go. If you don't, then you have to build empathy into the team. I found the book Practical Empathy to be helpful. But basically, if you treat someone with empathy, they can't help but start to feel empathy for you. I think we're wired that way. If your team doesn't have a high level of empathy, be the first.
Not sure if you have read the post fully, but they seemed to have created this architecture review meeting precisely to overcome the problems you are talking about. The orgranization wide meeting is a way to make sure everyone sees the RFC, while the architecture meeting is an in-depth review with a small group of relevant people.
Many people would agree the latter is needed in an organization beyond a certain scale, but I think the former is extremely important and often over looked. It provides a channel for people to be aware of what's happening across teams and build organizational context. It's also a forum for people to express themselves on anything they might have an opinion on and help establish a sort of the zeitgeist for the group. I've seen the alternative where you work within tight silos disconnected from the rest of the org, and it gets pretty dreary.
> In reality, it was just a test of who had the most political pull.
Every society works that way. When a group has to accept a decision, the most powerful person in the group decides.
I don't understand why any educated person today would think meritocracy is a good thing. A child looking at Wikipedia for 10 minutes could see all the flaws in it, yet it keeps being touted as some idealized model of social contract. In reality it keeps white straight rich men in charge.
Amen. Really silly things happen too, like you go through all this process and then months later some hotshot asks “wait, why are we doing this?” when launch is imminent.
Like we all agreed this was worth doing all along, right up to this very moment, and now we’re not sure all of a sudden. All the approvals, all the documents written and decisions we went through were apparently not compelling to this person. We could have had this discussion and perhaps killed the idea in the crib, now we’re having this discussion about something we’re about to launch.
In other words the processes are a fig leaf, which as you say is fine, but then it doesn’t even do the job of the leaf.
It’s a bitter pill to swallow seeing this article. I am a former Squarespacer who left in the past year. All I will say is consider a few things carefully about this:
1. This introduces a lot of process, bureaucracy, and chances of gatekeeping politics. The whole “yes if” mantra just centralizes the political power to the hands of fewer senior leaders by cutting off legitimate blocking criticism from the rank and file.
2. Judge processes by outcomes. What has Squarespace done / shipped / accomplished in the last four or five years? I’m not alone in feeling like it’s a head-scratcher why the company would pay the most senior engineers to erect slow moving bureaucratic decision-by-committee disempowerment structures like this, and then turn around and act like that’s a good thing or constitutes what you’d want to see as productivity from staff or principal engineers.
While I’m a proponent of writing things down, not surprising peer teams, pulling blockers or conflicts forward quickly … I don’t think templates and councils help more than they hurt, and often domain experts in different teams need empowerment to just totally bypass that time-wasting crap and do what needs to be done. If they don’t uphold the responsibility to do due diligence on review or finding conflicts, fire them or take them off the project, but don’t make everyone wear design-by-committee training wheels and act like that’s non-dysfunctional good engineering.
YMMV of course, but I reckon it’s really worth it to ask what is the real engineering output, speed, productivity following from this kind of large process overhead.
Full democratization/decentralization of engineering decisions is how you get into a quagmire of technical debt that can eventually kill your engineering teams. Some decisions need to be made by a group that is accountable overall to the health of the technical implementation. Like you can't be letting people use 5 different message bus solutions and expect your devops/support people to manage all of them effectively. Someone has to maintain sanity.
> Approvers say “yes” or “not yet.” We want to encourage constructive comments, so the template doesn’t suggest “no.” Instead of just vetoing an idea, it’s better to be clear on what it would take to approve it.
I'm not trying to be facetious, but what if the idea really is a No? Not every change is for the better. What if someone proposes "we should rewrite X in Y language," or "we should use Z database instead." Sometimes you need to say "No."
Otherwise you end up with feature creep, endless rewrites and refactoring, code churn, and a bunch of other issues.
Yes we should rewrite it in language Y if everyone on the team is comfortable with the language, it provides nonfunctional benefits, and has potential to drive business value.
It’s just about acknowledging the conditions that would make an idea a good one. All ideas are good in a specific context. Instead of assuming everyone’s aware of the current context, state the ideal context for an idea.
But just as ideas exist in a context, so does the decision. If your team (for example) doesn’t have everyone know the language, then “yes if (false)” is just “no”. Businesses move and conditions change so keeping this “no-go” idea around as a “yes if” just means that people will try to resurrect it later. It may be a spurious idea, but you’ve given it a false blessing by attaching a conditional approval.
"Yes if you can solve the halting problem" is better than a "no". It's "no, and there are mathematical proofs why it will never be a 'yes'".
More realistically, you're more likely to hit "yes if there is compelling evidence that $choice is better than $status_quo with respect to $criteria". Sometimes that sort of evidence is expensive-to-impossible to demonstrate such that it's basically a "no", but again, it's a much more specific "no".
I'm all for positive language and this sort of feedback. But one needs to keep in mind that this sort of setup can lead to a lot of Yak shaving. So there should be somewhere/someone in the process that addresses the value vs Yak shaving costs.
> What if someone proposes "we should rewrite X in Y language,"
Yes, if 50% of our development team is proficient in Y, X is a bottleneck or EOL, rewrite will solve significant problems.
> we should use Z database instead.
Yes, if Z is widely used in our stack, Z will realize advantages that cannot be achieved with the current database, Z is required to solve upcoming problems.
That's how I take it, at least: You shouldn't say "No," you should say "this is why not."
The "no" is bc challengeable reasons, which are useful to be explicit about
"This would take resources away from delivering X and limit devs ability to collaborate, so we need a way to motivate the budget precedence / growth and a way to afford 50% + 6mo per dev productivity hit around retraining"
"Database x is a VC co toy that risks dying, acquisition by Oracle, and losing customer data when we do not have time for self-inflicted nonsense. We need ways to mitigate the risk such as $10M in NIH DB dev budget in case the likely case happens. We already spent our innovation budget for X, so need more to motivate doing it again on Y."
Less than gate keeping, this is a great way to help mature early/mid career devs / job hoppers who haven't had the chance to live with many self-inflicted senior-level mistakes yet
The above comment is a good example of non constructive ways to say no.
Anybody could reject any RFC with the reasons you gave (especially with the vague “bunch of other issues” catch all).
So I’d say, yes, you can say no, if there’s specific, written down, largely agreed upon principles or goals. The thing is, when a no is in reference to specific principles and goals, it’s no longer a no. It’s a “yes if it fits with XYZ. Can you explain how this works because I can’t see it.”
If it’s a no in reference to vague reasons, it’s just gate-keeping. I suppose I can say yes to gate keeping if you can find a way to be encouraging to the RFC author and value the efforts and thinking that went into it. Otherwise gate keeping breeds frustrated complacency.
>If it’s a no in reference to vague reasons, it’s just gate-keeping.
What if it is yes with vague reasons? Why "yes" should be easier than "no" ? It's potentially extra work and extra code to manage. If something is vague in it's usefulness enough that it doesn't get yes with good reasons it should probably go back to drawing board
"This proposal is vague in its usefulness" is the same thing as not adhering to specific, written, agreed upon goals and principles. So you could say "yes if this becomes more specific in relation to X".
However, this situation can also easily arise where the goals and principles are vague, and in that case it's ultimately means that its more of a vague reasons to say no than a vague proposal.
The "it's potentially extra work and extra code to manage" is, imo, another wildcard vague reason to say no. It can be used to deny essentially any proposal anybody makes. It's Machiavellian. In a cooperative teamwork oriented environment, those things are given budgets so there can be some wiggle room for each proposal to progress.
I get your point, and they do address this later on in the piece kind of. It sounds like it's perfectly valid to say no, especially after reviewing the document and their architectural meeting where they discuss it more in depth. But they're also not trying to actively encourage saying no in the document, instead encouraging responses that points to how to get to a solid idea.
So saying "no" isn't forbidden, but it isn't the default either. At least that's my interpretation.
I had the same reaction but thinking more I’m actually kind of surprised by the elegance of “yes/not yet.”
One of the best questions you can ask someone (including yourself) when in disagreement is “what new data/fact/different condition would convince you of my position?” The “not yet” framing kind of sneaks that question in via just two words! “Not yet” implies you must define the criteria by which it turns to a yes.
Yeah. Not everyone who thinks they have a good idea has one, and the way this is structured it's very, very time-consuming to defeat a legitimately bad proposal. It feels a lot like when a conspiracy theorist says "Bildenberg cabal is made of hollow earth lizard people" and your only retorts are "yes" or "I'm not yet convinced".
This is correct. The arguments for taboo-ing the word "no" seem to be rooted in conflict aversion. Powerless people fear pushback and powerful people fear taking a stand.
I wish articles like this would follow up on it 3 years later to see what happened. Things always backslide unless you have a cheerleader for the process until it becomes part of the culture.
At work I use a pretty similar template for my design docs. Lately I have started to include a “strategy” section earlier in the design doc, so the first part of the design doc is giving all the background and context then the next section sets out the requirements and goals. Then you give the future vision that everyone can agree to. Then the next section gives a strategy of how to accomplish the vision e.g “we will take incremental steps first” vs “we will re-architect the service.” Then finally the next section is where the design starts, now that we agree on the future vision, requirements and our overall approach to the project we can talk about specific engineering choices and the plan to accomplish those.
I wanted to ask a question to other frontend folks, have you been able to get a good design doc process adopted by your team?
I have had difficulty because a lot of times FEEs think that their projects are too “simple” but then when I look at their code they have made design decisions that we could have discussed much earlier on if they had actually made a design doc. I also see FEEs going down wrong paths and re-learning things themselves that we could have addressed in a design review. What does your process look like for design reviews for Frontend?
We adopted a very very similar RFC framework for some projects back in 2020 and those project, now three years later, are the ones that have the least problems. Even though they took a little longer to get started.
What's that old army adage? "An ounce of sweat prevents a gallon of blood."
There's a huge swath - probably an overwhelming majority of the time - of good faith situations where it works well. That is, the suggestion isn't intended to be adversarial, and is otherwise plausible.
A major problem however is if there's not a collective understanding that "yes, if" is not a suggestion or consent for that condition to be actualized.
So it really should be "yes, if ..., however, that situation is very rare/cost prohibitive/high risk, so should be avoided"
There needs to be some way to unambiguously signal that something is a no-go, even if there are situations where it would be a good idea.
I run a RFC process for an open source project and it works quite well.
I do not love the “Yes, if” attitude though. Putting a condition on something really hinders the creative process. You end up designing by committee or rallying behind the “approvers” which makes for bad design in my opinion.
In comedy improv, there is a concept of “yes, and”. It is much more empowering. You accept the reality as is and build on top of it. If the changes get picked up, then it was meant to be. It helps foster trust in each other leading to great organic discovery on the way to being accepted.
I like the template poster there, as it reminds you of pertinent points without having too much boilerplate to fill in.
"Yesifs" are a handy way to avoid back/forth and shoe trust. They are also good in code reviews, or many gated places. If people use the yes if to do the yes and ignore the if, then there is of course a problem. Some places I have worked I feel like the team would have done that because of culture.
I've worked at two companies with RFC processes. At both companies, the processes were -- and I hate to use such a strong word -- a total sham.
I suppose the processes were crafted to look like some egalitarian meritocracy. In reality, it was just a test of who had the most political pull. If you had management on your side, your "RFC" was effectively law and dissenting voices were effectively career suicide.
Which, you know... fine. I understand that choices are made based on cliques and political capital rather than anything else. Cool. That's okay! Sucks sometimes, but that's how the world works. Hopefully your org structure itself is at least something of a meritocracy, so that edicts from on high are of a generally high-enough quality. If they get it right-ish often enough, your org will be okay... probably.
But what really rankled me was the fact that the RFC processes amounted to some kind of elaborate cosplay so that, I guess, folks could pretend that there was some sort of healthy collaborative process. If you're going to let a few "popular and management-blessed" engineers make all the decisions, fine, but don't add insult to injury by pretending otherwise.
I've heard similar things from others, elsewhere. I hope there are at least a few companies dedicated to being something better.