Summary: Based on lean thinking, we can see a logical direction for DevOps, which is to move to the business side. Business is the source of product development and operation, and a complete value stream must start from the source. This is not a prediction, but a fact that is happening.

Editor’s note: this article from ali greetings effect produced by the team, the alibaba conversation practice, guide to scan qr code above or go to: developer.aliyun.com/topic/devop…

Author: He Mian, senior technical expert of Aliyun Cloud Effect

When it comes to DevOps, we have to start with Agile and lean development. DevOps builds on them, borrows their methods, ideas, and develops and refines their practices.

Rise of Agile software development

The practice of agile software development first emerged in the 1990s. At that time, a number of lightweight software engineering methods and frameworks were born. Their common feature was that they followed an evolutionary and iterative model, and the process was more lightweight and flexible than traditional software engineering. Scrum and ExtremeProgramming are the most successful and influential in practice. Both are iterative and incremental software development frameworks, the difference being that Scrum covers only management practices, while Extreme Programming covers both engineering and management practices.

In the 1990s, the popularity of PC software and the emergence of fourth-generation programming languages, as well as the rise of the object-oriented and design pattern movement, allowed the development of small projects to flourish. At the same time, the rise of Internet applications and open source communities, different from the traditional development model has emerged, outstanding individuals in the program development has been highlighted.

All of these factors give unconventional development methods fertile ground to experiment with. As a result, on the one hand, quality problems kept emerging, which in part prompted the flourishing and popularization of CMM/CMMI derived from total quality management system. On the other hand, there are also many effective practices different from traditional methods, so that the industry sees new possibilities. The agile movement is now ready to emerge, which is not only a rebellion against tradition, but also a norm for wild growth.

In February 2001, 17 representatives of lightweight software engineering methods, including the inventors of Scrum and Extreme Programming, gathered at snowbird Ski Resort in Utah. After two days of meetings, the agile Manifesto (see Agilemanifesto.org/) was released, stating that they…

The agile concept emerged in 2001 at a good time. At the time, traditional methods were becoming unwieldy without solving the software crisis; On the other hand, mankind is entering the Internet era, and the software industry is rapidly upgrading its requirements for responding to changes and innovation. This is the more fundamental reason. After all, demand is the best booster for the development of the industry.

After the Agile Manifesto was published, Agile became a movement that was rapidly popularized and adopted. However, early Agile focused on the R&D delivery phase. From a business perspective, its goal was to help products and r&d teams improve their agile responsiveness, i.e., “deliver value earlier and be more flexible to change.” However, in the context of enterprise digital transformation, IT not only ensures product development and delivery, but also system deployment and operation. DevOps is a legacy of agile development and an addition to operations, but DevOps is more than just a superposition of development and operations, as we’ll get to later.

Emergence of lean product development

Lean thinking originated in the field of production and manufacturing, and originated from Toyota’s management and engineering practices in product manufacturing. A 1988 Sloan Management Review paper, “The Triumph of Lean Production Systems”, compared western and Toyota production methods with huge gaps in efficiency and quality, challenging the myth that scale brings benefits. Since then, Lean has entered the western vision and gradually become an important part of modern management.

Lean Thinking defines lean as: a new way of thinking about the effective organization of human activities, with the goal of eliminating waste in order to deliver more useful value. The book further summarizes the 5 principles of Lean, which are also the 5 steps of lean implementation:

1. Define value: Define value from the user’s perspective and describe it as a specific product or service;
2. Identify value Streams: Identify and map the process steps that create value and eliminate steps and activities that do not add value to the user;
3. Keep value flowing: Keep user value flowing through process steps so that it flows continuously and smoothly to end users;
4. User value pull: pull the flow by user value to avoid waste that does not bring user value;
5. Refine: Repeat steps 1 to 4. Pursue perfect value and value flow, eliminate all waste in the process.

At this level of abstraction, lean thought goes beyond the manufacturing industry where it was born and has a profound impact on various industries, such as lean government, lean hospital, Lean leadership, Lean service industry, lean supply chain, lean education, etc., which also includes product development. In fact, the mainstream agile development methods are directly influenced by lean thinking and follow the basic principles of Lean.

At the same time, lean product development has evolved rapidly as an independent practice that focuses on two goals — the value delivery process and the value itself.

First, focus on the value delivery process. The representative one is “Lean Kanban Method”, which was developed by David Anderson based on his own practice around 2006 and systematically summarized in his book Kanban Method published in 2010. Kanban method is the specific application of lean ideas in software development. It starts with visualizing the end-to-end value stream of requirements delivery, improves the flow efficiency of requirements through system practice, and ensures the process quality of flow to achieve end-to-end system improvement.

The essential change of lean practices represented by “Kanban method” is to shift focus from efficient use of resources to efficient flow of value. This also drives users from local optimization to end-to-end global optimization.

Second, focus on value itself. One of the most representative is “lean entrepreneurship”. The practice of lean entrepreneurship was first developed by Steve Blank based on his own and other Silicon Valley entrepreneurial practices. Eric Ries’ book, The Lean Startup, provides a systematic summary of the concepts and practices of the Lean startup, and rapidly popularized the concept of the Lean startup.

Lean entrepreneurship sees entrepreneurship as a hugely uncertain process whose biggest waste is delivering things that don’t work (that don’t solve user problems, or lead to business success). To this end, it integrates value exploration and discovery into the product delivery process, introducing the famous “development-measure-learn” cycle. The cycle begins with untested concepts about markets and products. Next, the first step of the cycle was to develop a minimum Viable Product (MVP, Minimal Viable Product) to validate the concept. Step 2: Collect market and user feedback based on the minimum feasible product, and obtain measurement data; Step 3: Test hypotheses with data, confirm or falsify them, and adjust them to produce empirical knowledge. Then, move on to the next cycle of exploring business models and product features.

In fact, the lean Startup book defines “startup” as creating new businesses and products in an uncertain environment. And “uncertainty” seems to have become common today in the areas of IT environment, and therefore, the MVP, “development – measuring – learning” concept such as cycle IT innovation has become a recognized practice, and around the lean startup developed a complete system of innovative practice, such as lean data analysis, customer development, lean design delivery, etc.

Explore and discover effective value and allow it to flow smoothly. Around these two goals, and following lean thinking, lean product development has developed into a systematic practice. The influence of Lean thinking on DevOps is also very fundamental, and the DevOps Three principles are completely followed by Lean thinking.

The birth of the conversation

Initially, both agile and lean communities focused only on development side practices and behaviors, and operations did not become their focus. However, system development alone is not enough, the developed system must be immediately smooth deployment, and continuous and stable operation to achieve value. Traditionally, that part has been handled by operations.

From the perspective of value, development plus operation and maintenance constitute a relatively complete IT value chain. Of course, a more complete one should also include the business, which is a later story, but was not the focus of the early DevOps community. When DevOps was born, the problem IT solved was between development and operations, which was the most prominent issue affecting the IT value chain.

In traditional IT organizations, development teams (Dev) and operations teams (Ops) have different demands — development teams (especially agile teams) seek change, while operations teams seek stability. There is often a conflict of interest between, for example, lean and Agile teams that focus on continuous delivery, and operations teams that emphasize change control for online stability. This creates departmental walls, which of course are not conducive to maximizing IT value.

In 2009, at the second Velocity Conference in the US, John Allspaw and Pauk Hammond from Flickr Gave a talk called 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr. In this presentation, Allspaw and Hammond role-play the various conflicts between development and operations. There are many golden phrases in the speech, such as “It’s not my code, It’s your machines!” “, which speaks volumes about the state of the Dev/Ops relationship. They then show how by removing barriers between development teams (Dev) and operations teams (Ops), they can work together to make software distribution and operation sustainable and efficient through tool and culture changes.

The presentation was a landmark event in the development of DevOps. It asks the right question — the gap between development and operations must be bridged in order to deliver and realize value faster — and suggests a series of changes in culture, tools, and practices that are needed to bridge the gap.

In the same year, Patrick Debois, an independent IT consultant in Belgium, saw this talk and was inspired to organize the first DevOpsDays. DevOps took off, the idea of DevOps caught on, and the tools and practices involved grew rapidly. The emergence of cloud native technologies represented by containerization and automatic scheduling has also greatly accelerated this process. Today, DevOps is one of the core capabilities of enterprise digitization, a fundamental requirement for IT delivery and operation.

Subsequently, in the Phoenix Project and DevOps Practice Guide, Gene Kim et al. summarized the three steps of DevOps implementation, which are:

Flow principle: Focus on the overall value stream of the IT system, optimize globally, and ensure the rapid flow of value from left (upstream) to right (downstream).

Feedback principle: Create feedback loops from left to right, shorten feedback cycles and amplify feedback. In this way, you can respond and understand internal and external customers more quickly, and gain the knowledge needed to improve in real time.

Principles of continuous experimentation and learning: Create a culture of risk-taking, continuous experimentation, and learning from mistakes, improving capabilities through experimentation, and improving system resilience.

Kim et al. believe that this three-step approach is the value and philosophy underpinning all other DevOps processes, practices, and that all DevOps patterns can be derived from these three principles.

A little digging reveals that the DevOps three-step approach is a reworking of lean principles. More specifically, they are concrete examples of lean principles in the context of IT development and operation. In fact, the foundation of DevOps shows the influence and application of lean principles.

conclusion

Looking back at the evolution of Agile, Lean, and DevOps, we can draw two conclusions.

First, DevOps is a natural outgrowth of agile development practices. The goal of agile development is to “deliver value earlier and be more flexible to change”. The Agile movement begins on the development side, but without change on the operations side, it will become a bottleneck and ultimately the goals of Agile will not be achieved. In order for agile practices to be of real value, development operations must be aligned.

Second, DevOps is a logical consequence of lean thinking applied to IT. The goal of lean product development is “smooth delivery of effective value”, while Lean thinking requires end-to-end system optimization and continuous improvement. Development and operation and maintenance are two important parts of the system. The three Principles of DevOps are the specific examples of lean thinking in THE field of IT development and operation.

Finally, starting with lean thinking, we can see the inevitable direction of DevOps, which is to move to the business side. Business is the source of product development and operation, and a complete value stream must start from the source. This is not a prediction, it is happening, most DevOps implementations have already included the business side, called BizDevOps, but the DevOps term has become so popular that we will continue to use the DevOps term, but by default it includes the business side.

With the development of DevOps, digital transformation has also become the consensus of the business community. Most enterprise digital frameworks regard DevOps as one of the core capabilities, and the influence of DevOps is also expanding, becoming the inevitable choice of enterprises striving to improve their digital capabilities. In the next section, we’ll look at the fundamental problems DevOps is trying to solve in the context of digital transformation.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.