Brief introduction:From a lean perspective, we can see the inevitable direction DevOps is going, which is to extend to the business side. The business is the source of product development and operation, and the complete value stream must start from the source. It’s not a prediction, it’s a fact that’s happening

Editor’s note: this article from ali greetings effect produced by the team, the alibaba conversation practice, guide to scan qr code above or go to: https://developer.aliyun.com/topic/devops, download the full version of e-books, understand ali ten years experience in the conversation practice.

The author: He Mian, senior technical expert of Ali Yunyun Efficiency

When we talk about DevOps, we have to start with agile and lean development. DevOps developed from them, borrowed from them, and developed and improved their practice system.

The rise of agile software development

Agile software development practices first emerged in the 1990s. At that time, a number of lightweight software engineering methods and frameworks were born one after another. Their common feature is that, compared with traditional software engineering, they all follow the evolutionary and iterative model and the process is more lightweight and flexible. Among them, Scrum and ExtremeProgramming are the most successful in practice and have the greatest impact. Both are iterative and incremental software development frameworks, the difference being that Scrum covers only management practices, while Extreme Programming covers both engineering and management practices.

In the 1990s, the popularity of PC software and the emergence of fourth generation programming languages, as well as the rise of the object-oriented and design patterns movement, allowed the development of small projects to flourish. At the same time, the rise of Internet application and open source community, different from the traditional development mode is constantly emerging, the role of outstanding individuals in program development has been highlighted.

All of these factors make it possible to experiment with unconventional development methods. As a result, on the one hand, quality problems emerge in an endless stream, which partly promotes the prosperity and promotion of CMM/CMMI from the total quality management system at this time; On the other hand, many effective practices that are different from the traditional methods have been produced, making the industry see new possibilities. The agile movement was emerging, both as a rebellion against tradition and as a norm against barbarism.

In February 2001, 17 representatives of the lightweight software engineering approach, including several inventors of Scrum and Extreme Programming, gathered at the Snowbird ski resort in Utah. After two days of meetings, released later have a huge impact of the agile manifesto (see http://agilemanifesto.org/)), the agile manifesto presented their common recognition about the concept of software development methods, it is also important they found the word agile to master these concepts.

The emergence of the Agile concept in 2001 could not have been more timely. At the time, traditional approaches were becoming more and more cumbersome without solving the software crisis. On the other hand, we are entering the Internet age, and the software industry is rapidly upgrading its requirements to respond to change and innovation. This is more fundamental because demand is the best driver of industry development.

After the Agile Manifesto was published, Agile became a movement that was rapidly spread and applied. However, early Agile focused on R&D delivery. From a business perspective, its goal was to help products and R&D teams become more responsive, that is, to “deliver value earlier and be more flexible to change”. However, in the context of the digital transformation of the enterprise, IT is not only important to ensure product development and delivery, but also system deployment and operation. DevOps inherits the agile development concept and complements the operations part, but DevOps is more than just a combination of development and operations, as we’ll see later.

The emergence of lean product development

Lean thinking originated from the field of production and manufacturing, and originated from Toyota’s management and engineering practices in product manufacturing. A 1988 Sloan Management Review paper, “The Triumble of the Lean Manufacturing System,” challenged the myth of the benefits of mass production by comparing the vast gap in efficiency and quality between Western and Toyota-style production. From then on, Lean began to enter the vision of the West and gradually became an important part of modern management.

The book Lean Thinking defines Lean as a new way of thinking that effectively organizes human activities with the goal of eliminating waste in order to deliver more useful value. The book further summarizes the five principles of Lean, as well as the five implementation steps of Lean:

1. Define value: define value from the user’s perspective and describe it as a specific product or service;
2. Identify Value Streams: Identify and map process steps that create value, eliminating steps and activities that do not add value to users;
3. Keep the value flowing: Keep the user value flowing through the process steps so that it flows continuously and smoothly to the end user;
4. User value pull: the flow is pulled by user value to avoid the waste of user value;
5. Keep improving: Repeat steps 1 to 4. Pursue perfect value and value flow, eliminate all waste in the process.

At this level of abstraction, Lean thinking transcends the manufacturing industry where it was born and has a profound impact on various industries, such as Lean Government Affairs, Lean Hospital, Lean Leadership, Lean Service Industry, Lean Supply Chain, Lean Education, etc., which also includes product development. In fact, the mainstream agile development methods are directly influenced by Lean thinking and follow the basic principles of Lean.

At the same time, lean product development has also grown rapidly as an independent practice system that focuses on two objectives — the value delivery process and the value itself.

First, focus on the value delivery process. The most representative one is “Lean Kanban Method”, which was developed by David Anderson around 2006 based on his own practice and systematically summarized in the book Kanban Method published in 2010. Kanban method is the concrete application of lean thinking in software development. It starts by visualizing the end-to-end value stream of requirements delivery, improves the flow efficiency of requirements through systematic practices, and ensures the quality of the flow process, thus achieving end-to-end system improvement.

The essential change of this kind of lean practice represented by Kanban method is that it shifts from focusing on the efficiency of using resources to focusing on the efficiency of flow of value. This also drives users from local optimization in the past to end-to-end global optimization.

Second, focus on value itself. One of the more representative is the “lean startup”. The practice of Lean Startup was originally developed by Steve Blank based on his own and other entrepreneurial practices in Silicon Valley. In the book Lean Startup, Eric Ries made a systematic summary of the concept and practices of Lean Startup, which quickly popularized the concept of Lean Startup.

The Lean Sttups view entrepreneurship as a hugely uncertain process, where the biggest waste is delivering stuff that doesn’t work (it doesn’t solve user problems, or lead to business success). To this end, it integrated value exploration and discovery into the product delivery process, and put forward the famous “development-meter-learning” cycle. The cycle begins with an untested concept about the market and the product. Next, the first step in the cycle is to develop a Minimal Viable Product (MVP) to verify the concept. Step 2: Collect market and user feedback based on the minimum viable product, and obtain measurement data; Step 3: Test hypotheses with data, confirm or disprove them, and adjust them to produce empirical knowledge. Then, enter the next cycle, continue to explore the business model and product function design.

The impact of the Lean Startup extends far beyond a startup. In fact, the Lean Startup book defines the term “startup” as the creation of a new business or product in an environment of uncertainty. And “uncertainty” seems to have become common today in the areas of IT environment, and therefore, the MVP, “development – measuring – learning” concept such as cycle IT innovation has become a recognized practice, and around the lean startup developed a complete system of innovative practice, such as lean data analysis, customer development, lean design delivery, etc.

Explore and discover effective value and let it flow smoothly. Around these two goals and following lean thinking, lean product development has evolved into a systematic practice. The impact of Lean thinking on DevOps is also fundamental, and the three DevOps Principles follow Lean thinking perfectly.

The birth of the conversation

Initially, both the Agile and Lean communities were focused on development-side practices and behaviors, not operations. However, system development alone is not enough. The developed system must be deployed smoothly in time and run continuously and stably to realize value. Traditionally, this has been the responsibility of operations.

From the perspective of value, development, operation and maintenance constitute a relatively complete IT value chain. Of course, a more complete version should also include the business, but that’s for another day, and it wasn’t the focus of the early DevOps community. When DevOps was born, the problem IT solved was between development and operations, which was the most prominent problem affecting the IT value chain.

In a traditional IT organization, development teams (Dev) and operations teams (OPS) have different aspirations — development teams (especially agile teams) want change, and operations teams want stability. There is often a conflict of interest, for example, where lean and agile teams focus on continuous delivery as a goal, while operations teams emphasize change control for the sake of on-line stability. Departmental walls are built, which of course is not conducive to maximizing IT value.

In 2009, at the 2nd Velocity Conference in the US, John Allspaw and Pauk Hammond from Flickr gave a presentation called 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr. In this presentation, Allspaw and Hammond bring to life the various conflicts between development and operations in a role-playing manner. It’s not my code, It’s your machines! “, which reflects the current state of the relationship between Dev and OPS. They went on to show how by breaking down the barriers between the development team (Dev) and the operations team (OPS), both sides can work together to make the release and operation of software sustainable and efficient through tool and cultural changes.

The presentation was a landmark event in the evolution of DevOps. It asks the right questions — the gap between development and operations must be bridged in order to deliver and realize value faster — and offers solutions — a series of changes in culture, tools, and practices are needed to bridge the gap.

In the same year, Patrick DeBois, an independent Belgian IT consultant, saw the talk and was inspired to organize the first Devopsdays. DevOps was officially launched, the idea of DevOps became popular, and the tools and practices around IT grew rapidly. During this period, the emergence of cloud native technologies represented by containerization and automatic orchestration also greatly accelerated this process. Today, DevOps has become one of the core capabilities of enterprise digitalization, a fundamental requirement for IT delivery and operations.

Later, in Project Phoenix and DevOps Practice Guide, Gene Kim et al. summarized the three-step approach to DevOps implementation. They are:

Flow Principle: Focus on the overall value stream of IT systems, optimize globally, and ensure the rapid flow of value from left (upstream) to right (downstream).

Rule of Feedback: Create feedback loops from left to right, shorten feedback cycles and amplify feedback effects. This enables faster responsiveness and understanding of internal and external customers, and instant access to the knowledge needed for improvement.

Principles of continuous experimentation and learning: create a culture of risk-taking, continuous experimentation, and learning from mistakes, improving competence through trial and error, and improving the resilience of the system.

Kim et al. believe that these three steps are the value and philosophical foundation for all other DevOps processes and practices, and that all DevOps patterns can be derived from these three principles.

If you look a little further, the DevOps three-step approach is a mirror image of Lean principles. More specifically, IT is a concrete example of Lean principles in the context of IT development and operations. In fact, the foundation of DevOps is the influence and application of Lean principles.

conclusion

Looking back at the evolution of Agile, Lean, and DevOps, we can draw two conclusions.

First, DevOps is a natural evolution of agile development practices. The goal of agile development is to “deliver value earlier and be more flexible to change.” The Agile movement started on the development side, but if there is no change on the operations side, it will become a bottleneck and ultimately the Agile goals will not be achieved. In order for Agile practices to have real value, the linkage between development operations and maintenance is imperative.

Second, DevOps is the inevitable result of the application of Lean thinking in IT. The goal of lean product development is “smooth delivery of effective value”, while lean thinking requires end-to-end system optimization and continuous improvement. And development and operation and maintenance are two important parts of the system. The three principles of DEVOPS are the concrete examples of lean thinking in IT development, operation and maintenance.

Finally, from a lean perspective, we can see the inevitable direction DevOps is going, which is to extend to the business side. The business is the source of product development and operation, and the complete value stream must start from the source. Most DevOps implementations already include the business side as BizDevOps, but the term DevOps has become popular and we will continue to use it, but by default it includes the business.

With the development of DevOps, digital transformation has also become a consensus in the corporate world. Most enterprise digital frameworks regard DevOps as one of the core capabilities. The scope of influence of DevOps is also expanding, and it has become an inevitable choice for enterprises seeking to improve their digital capabilities. In the next section, we examine the fundamental issues DevOps is trying to address in the context of digital transformation.

Free download of Alibaba DevOps Practice Guide

Alibaba partners and industry leaders recommended, He Mian, Chen Xin and other 17 Ali senior technical experts jointly produced, Alibaba ten years of DevOps experience precipitation summary, Alibaba DevOps implementation practice.

To: https://developer.aliyun.com/topic/devops, download the full version of e-books.

Copyright Notice:The content of this article is contributed by Aliyun real-name registered users, and the copyright belongs to the original author. Aliyun developer community does not own the copyright and does not bear the corresponding legal liability. For specific rules, please refer to User Service Agreement of Alibaba Cloud Developer Community and Guidance on Intellectual Property Protection of Alibaba Cloud Developer Community. If you find any suspected plagiarism in the community, fill in the infringement complaint form to report, once verified, the community will immediately delete the suspected infringing content.