What is consensus and why is it important in engineering organizations?
Engineers often need to solve ambiguous problems. Typically there is not a “correct solution” to engineering problems. Sometimes there is no good solution at all so a bad solution must be implemented. This is a difficult situation to be in as an engineer and it is easy to become paralyzed by fear and uncertainty.
When faced with an ambiguous and complex problem, the best an engineer can do is make sure that their organization is broadly in agreement on the approach that should be taken. The broad agreement of an organization is consensus.
If the organization agrees on the solution to a problem, it no longer matters if the solution is a good one or a bad one (and usually the organization will not find out until much later). The engineer can commit to the solution and continue making forward progress. In engineering organizations, speed is important so engineers must solve problems quickly. Some problems require consensus so engineers must be able to effectively and decisively drive consensus.
When you can avoid driving consensus
Not all solutions to problems require consensus. Consensus is an expensive process so if it can be avoided it probably should be avoided.
An engineer can sometimes avoid driving consensus when they are the most knowledgeable about a problem. If it is unlikely that the organization can contribute meaningfully to the solution then it may not be worth it to complete the consensus process. Driving consensus can also be an opportunity to educate the organization, so carefully consider if the opportunity to educate is more important than the opportunity to move faster.
An engineer can avoid driving consensus when they are comfortable taking full responsibility for the outcome of the solution. When consensus works properly it is a good way to avoid blame falling on an individual. If an engineer is willing to accept the possibility of blame falling on them alone, they can avoid driving consensus. This is a common reason to avoid consensus for small decisions.
As engineers become more experienced and become more comfortable in the organization, they will make larger decisions without consensus. Conversely, inexperienced engineers may need to drive consensus for relatively simple problems. It is typically better to drive consensus when it is not needed than to not drive consensus when it is needed.
How to drive consensus
When an engineer has identified a problem that requires consensus, their first step is to create a proposal. That proposal should be highly opinionated. It should present a comprehensive solution to the problem. It is acceptable to have a section in the proposal for “alternatives considered” where different solutions are considered and rejected, but it is not acceptable to have multiple solutions in the main sections of the proposal. The reader of the proposal should only need to evaluate the proposed solution, they should not need to compare multiple solutions. The proposal should present the problem, a single solution and should justify why the solution should be accepted by the organization. Remember that driving consensus has nothing to do with finding a correct solution, so it is expected to propose a solution with clear flaws.
Once a proposal has been created, the proposer should designate a single approver. That approver decides when consensus has been reached on the solution to the problem. The approver should have some way of updating the proposal to mark it as approved. The approver should typically be a senior member of the organization and should have some familiarity with the subject matter of the proposal. The approver should be invested in solving the problem so they are motivated to reach a timely solution. The approver should feel comfortable soliciting the feedback of other members of the organization on the proposal.
The proposal must also be sent out to the broader organization. This could be via email or a forum. Any member of the organization who is interested should be able to easily find the proposal and should have the ability to make comments on the proposal. By making proposals public in the organization we can assume that anyone who does not leave a comment finds the proposal acceptable.
The approver should give time for the broader organization to leave comments and should leave some of their own. Comments can include requests for clarification, questions about how the solution will work, and suggestions for ways to improve the proposal. The proposer should seek to resolve the comments by replying to comments and making updates to the proposal. Typically the approver should not approve the proposal until the majority of comments are resolved. Once the proposal has been approved, implementation can begin.
After approval, the proposal is not likely to be edited significantly. Typically proposals represent a decision at a specific time, not as living documentation that is continuously updated.
Common issues with consensus
There are a few ways that driving consensus can falter or fail.
There is disagreement on the solution. If the organization does not agree on the solution, the proposer may get a lot of comments on their proposal in the section describing the solution. Ideally, a clear disagreement will be reached, but if indirect communication occurs, it may not be as clear that there is a disagreement. If a resolution to the disagreement is slow, the proposer should schedule a meeting between the proposer, the approver, and the disagreeing party. Everyone in the meeting should make their best attempt to do what is best for the organization and should recognize that a quick resolution is important. In some cases, the organization will decide that the problem does not need to be solved at all.
There is disagreement on the problem. If the organization does not agree on the problem the proposer may get a lot of comments on their proposal in the section describing the problem. If the problem is an engineering-driven problem then consensus can be reached through a similar process to the above section on solution disagreement. However, problems can also be driven by the business and that can be more complicated to navigate. If engineers disagree with a problem motivated by the businesses that could be a sign that the engineers are out of touch with the business problems or it could be a sign that the business is out of touch with the engineering problems. Regardless, it is usually a good next step to set up a meeting with the approver, the disagreeing party, the proposer, and a representative from the business. Ideally, a resolution to the disagreement can be reached and a process can be set up to keep the business and the engineers more closely aligned.
The proposer is unable to take feedback on their proposal. It is a problem for a proposer to become overly attached to their proposal and to consider feedback on the proposal to be personal attacks. In a situation like this, the proposer needs to recognize that their job is not to produce a solution, it is to produce consensus. The content of the proposal does not matter, what matters is the agreement of the organization.
What it looks like to succeed at driving consensus
When driving consensus goes well, ambiguous and complex problems can be solved quickly and with low overhead.
Successful consensus will not require many meetings. If there is no significant dissent then no meetings will be needed. The proposer can simply send out the proposal, respond to comments on the proposal and it will be approved by the approver quickly. If there is significant dissent then at most one or two meetings should be needed to reach a resolution.
Successful consensus will result in a proposal becoming better than it was at the start. By driving consensus the entire knowledge base of the organization comes into play. If engineers are invested in the organization they will be motivated to review proposals to have a say in the evolution of the software architecture and can apply their knowledge to improve the proposal.
Successful consensus will improve the organization. If the organization agrees on a solution and is wrong, then that is an opportunity for organizational growth. If the organization agrees on a solution and is right, then that is an opportunity to celebrate and to aim even higher.