It ‘s Microservices… It ‘s docker… It ‘s organization structure

Microservices and containers are the undisputed themes of this era, with teams and companies scrambling to try them out, but often finding it difficult to resonate across the organization. In this paper, we attempt to uncover the changes in organizational structure that are reflected behind microservices and container technologies, and the adverse constraints that organizational structure imposes on the implementation of microservices and container architectures. Finally, we use the INVEST principle to look at the characteristics of organizational structures that support such loosely coupled architectures. Hope to help confused enterprises and organizations rethink their own micro services.

Microservices and Docker have become the new darling of the software industry in the past year, and every time I go to an event about them, I can’t help but feel Mr Conway standing behind me grinning “I told you so”.

Although Martin Fowler was careful to warn of the costs when he “defined” microservices, it seems that the benefits of microservices are so compelling that most software development organizations and enterprises have adopted microservices as an architectural approach of choice for the future, Everyone thinks I’m that tall! .

(Martin Fowler’s metaphor for the high engineering practical capabilities that accompany microservices architectures.)

As container technology reaches the critical point of production and application, this chemical effect seems to be ready to explode, and we seem to see a different software microservices market unfolding in the future.

In this market, there will be a platform like Taobao, which sets up an online mall for small and medium-sized service sellers. Buyers can search and purchase a wide variety of services according to their own needs. Online or binary, code quality and automation coverage have become important criteria for rating similar services.

Killer services such as blockchain or quantum cryptography could become crown selling services. Finally set off a big wave of procedures ape open micro service store upsurge ~ very looking forward to that is how the seller show and buyer show ah!

This almost sci-fi description may be suitable only for talk on wechat, but the popularity of micro services and container technology is no accident. Mr. Conway used his law to reveal a deeper truth:

This is not a revolution in technical architecture or infrastructure, but a necessity to maintain organizational flexibility.

In other words, software development organizations or enterprises are beginning to realize that all management and technical practices must serve to maintain the highest organizational flexibility possible. Why keep flexibility “as high as possible”? Don’t iron and steel troops and stable rules and regulations also create so many successful organizations and enterprises in history?

On the highest possible organizational flexibility

So I added the adjective “software development,” but now of course we have a broader term: technology companies. We usually think that the technological content of products or services is relatively high, with core competitiveness, can constantly launch marketable new products, and constantly expand the market of enterprises for science and technology enterprises.

In the era of software we live in, a lot of technology companies are involved with software. But historically we can think back to the age of mass production when steelmaking was high tech, and the information age when emailing was high tech. The first two waves of “tech companies” do not strike us as flexible: the steel giants are reminiscent of the country’s call for productivity for all; The giants of the information age, such as Microsoft and IBM, are associated with large, thousand-strong industrial software development teams, each deployment with a team of experts. So why do tech companies now have to be flexible and as high as possible?

So again, we’re going to use conway’s law to make inferences, and Conway’s law says

“The design (architecture) of a product or system is constrained by the communication structure of its production organization itself,”

In other words, if you have a front end presentation team, a back end services team, and a database team, we can be sure that the system will have a back end and a database design. This is a pessimistic law in itself, so the front team must communicate three times when new requirements are discovered, the front and back team are concerned about the impact of new requirements on their own architecture, and the database design is concerned about the impact on the existing data structure, which always ends up with an awkward solution in the dispute between all parties.

Many teams have long been used to this kind of pain, and the frequency of change in the digital age has brought it to a head. For example, it took 71 years for Shougang to reach 10 billion yuan output value, 13 years for Lenovo, and only 3 years for Xiaomi of this era! And this year xiaomi is no longer the tech upstart standing at the top of the wave.

So Old Mr. Conway said,

If you want to keep your products competitive, you need to keep your organization flexible.

Someone once argued to me, “What we are doing is a back-end system in the digital age. We are not directly facing the market. The demand is stable. So I pointed to their system with tens of thousands of lines of code and said, “At least 30 percent of your code is redundant, which is another consequence of organizational inflexibility.”

We’ll narrow down to software development for a moment, and it’s interesting to note that in our industry, no two developers can write the same code for the same requirement (with the possible exception of Hello World). Even more, WHEN I find two programmers using the same variable names I suspect they are copying. This shows that software development from requirements to code is actually doing design, different people design things will be different, like everyone’s signature.

The design even extends to later software testing and deployment, where the same application may behave differently in different network topologies. Organizations that seek stability want to end the highly uncertain activity of design as soon as possible so that they can improve efficiency through standardization.

Even in today’s mainstream agile development, many teams are still architects “drawing” and coders stacking code. So organizations like this quickly find themselves stuck in binary. I often tell teams that you lack “code responsiveness.” Responsiveness requires an organization to be flexible and able to navigate the uncertainties of design activities from front to back.

In summary: The market is changing rapidly in the digital age, and Mr. Conway has shown us that it is important for technology companies to maintain organizational flexibility in this era. The demand for organizational flexibility is compounded by the uncertainty created by strong design in software (broadly new technology) itself.

Therefore, in this era, we see that already successful enterprises such as Google and Haier begin to reform their organizational structure aggressively. Such extreme pursuit of flexibility contributes to the core competitiveness of these organizations to maintain the leading level of the market. There is no doubt that the advantages of microservice architecture also reflect the flexibility of organizational structure, and the application of container technology reduces the operating cost of this loose market structure, just as the emergence of Taobao platform has built a basic trading platform for millions of sellers and buyers.

Flexible and scalable containerized computing resources coupled with loosely coupled microservices architecture will certainly attract enterprises pursuing organizational flexibility to defeat Conway’s Law and maintain organizational vitality.

INVEST principles for organizational structure

We have discussed the necessity of maintaining organizational flexibility in the digital age. What principles should flexible organizational structure meet? Let’s use INVEST, a well-known requirements management principle in agile development, to examine how an organization can truly leverage the flexibility benefits of microservices architecture and container technology, or take another look at supporting the use of microservices architecture.

(Comparison of the two organizational structures)

Independent: Independent

Microservices teams should provide one or more services externally, and the services should be loosely coupled, so the teams behind them should also be relatively independent. Following conway’s law, if a large organization failed to differentiate the relatively independent of service oriented team, and finally provides the internal structure of the product or system and can not be simple service assembly, and will be we often say “pasta”, the internal structure entanglements that finally are getting slower and slower response to the new market demand. Easy to communicate: Holder

The structure of “small” service team inevitably leads to the marketization and communization of the whole organization. Without good communication between teams, it is hard to imagine any output in such an organization. Amazon is considered a successful example of the use of microservices architecture, and its 2Pizza team’s principles have become the industry’s benchmark.

But such service-oriented small team set the bottom of the formation of long-term running in good communication mechanism between team, even when we asked Amazon teams how to find other services or other team assist with requirements, the team cannot say the process mechanism of concrete, everything is very natural, all like we went to his familiar supermarket, You find your daily needs naturally.

Valuable: Valuable

There is no doubt that every team must be value-oriented. In fact, the proposal of agile development method pointed out the original sin of waterfall delivery mode divided by functional departments in the traditional way, that is, each functional department is not responsible for Output over Outcome (Output over Outcome). Such organizational structure is bound to cause a lag in response to market changes.

It is worth mentioning that value-delivery teams are often cross-functional. According to the architecture of microservices, the team needs to be responsible for the full lifecycle of services from requirements to deployment and operation (Outcome over Output). This is why the team must be “tall” in terms of basic engineering delivery platforms and practices.

Estimable: Estimable

Such a service team should have a very short lead time that can be accurately estimated and should come online on a regular basis, rather than the big bang model of the past, which has ranged from a few months to a year. Continuous delivery should be standard practice in such organizations, and it is the team’s shared responsibility to keep software systems in a releasable state at all times. Modern tech companies like Amazon and Netfliex have seen the engineering capabilities that have evolved under this structure translate into great success in business services.

Short: Small

As mentioned above, Amazon’s 2Pizza team is less than 10 people, and Scrum, the classic agile management framework, also recommends a team of 5 to 9 people. Therefore, small teams become a necessary condition for organizational flexibility. The Chinese saying, “A small ship is easy to turn around,” ingeniously reveals a little flexibility, but why not a little more? Like two people on a team.

It’s easy to see how the complexity of software development itself makes delivering value end-to-end impossible for a two-person team. In terms of overall organizational robustness, too small a team also increases the risk of a single point of dependency. Although it has not been officially confirmed, we found in our communication that there is service redundancy in micro-service organizations like Amazon, which ensures that risks can be properly avoided after the organization is divided into small teams.

Testable: Testable

In the face of high uncertainty in market conditions, we should get this matter straight. The traditional large functional organizational structure is often fault-tolerant, and the price of mistakes is that the whole enterprise goes astray, or a department loses the right to speak in the enterprise.

In an organization with high flexibility, it should be easy to conduct such “testing”. Enterprises are more able to use such a structure to carry out dynamic portfolio management, such as Google’s famous 7:2:1 investment ratio. The last achievement is to use the flexibility of the organization to conduct innovation testing. The result of testing is often a failure, but it is this constant testing that has created many famous “black swans” in Google’s history.

Break Conway’s law

Recently, many theoretical frameworks with LEAN as the key word have emerged in our field, including LEAN Enterprise, which I mentioned in my previous article. As a result, some friends have teased that they have started to stir up concepts again. I am very serious about making it clear that I do not want to lose concept, which is why I return to the concept demonstrated and developed in the last century: lean.

Lean from Toyota manufacturing summarizes a lot of principles and practices, but intentionally or unintentionally, Toyota has completed its own organization to build the continuous flexibility of this great achievement beyond other enterprises in the same period. The result is greater resilience in response to diverse demands.

If we use the INVEST principle to look at lean organization, you will soon find corresponding principles and practices. Even in the traditional industrial assembly line, Toyota is forming small teams (cell production) and completing “cross-functional” within small teams through multi-skill training of employees. Its core philosophy of continuous improvement (Kaizen) is a strong guarantee of value-oriented team work and a good cross-team communication culture.

In a sense lean broke Conway’s Law before it was defined!

Microservices and container technologies are undoubtedly important steps in engineering architecture to support organizational flexibility in this era. However, we should not forget that an organization has all the components, as mentioned in Lean Enterprise, The organization’s financial audit, human resources, procurement compliance and other functions how to effectively “micro-service” and how to jointly build a flexible “container” support platform still need your efforts!