A few weeks ago, I attended and spoke at the KubeCon EU conference. It was a large event with around 4,700 people, which reminded me of the OpenStack Summit in Paris in November 2014. Same crowds, manufacturers show up, programmers party… However, I found a fundamental problem: I met almost all the operations people or SRE engineers. Where are the application developers? Isn’t this complex infrastructure supposed to serve these people? Does the Kubernetes community really care about the needs of its users? So I can’t help but wonder: Is Kubernetes too complicated? Will its complexity hold it back?

I’m going a little too far in thinking that, but I don’t think it’s going to end up that way. Kubernetes has its own unique advantages over OpenStack. First, it is more scalable, enabling large-scale cluster scheduling in an environment with thousands of servers. Second, three major cloud providers have launched Kubernetes hosting offerings. In just four years, Kubernetes has almost become the de facto standard in cloud computing and infrastructure. However, the complexity of Kubernetes is still a problem for everyone. OpenStack has been declining since 2014. Will Kubernetes follow in OpenStack’s footsteps?

Kubernetes is designed to solve complexity and problems on Google’s scale. We want infrastructure to be self-healing, horizontal scaling, and declarative configuration (i.e., infrastructure as code). The problem is that the vast majority of applications and application developers don’t face these problems. Most applications are modest in size (both project size and user base), or they are in the early stages of market product research.

There is usually no need to add such complexity until the application has reached scale. In this case, a single database and application server may be a better choice, as long as they can handle your traffic load. If 99.5% of the time is available, with full operational alerts, that’s good for most applications and workloads. The complexity of building a distributed system and the delay in time to market is more than worth it.

For brand new applications, their biggest risk is not finding a target product or market. That is, these applications are deployed but never used. Their code was eventually abandoned because no one wanted to use the applications. This is the situation most application code is likely to face. Most of my career has been spent iterating over code and features, trying to find the right set of application features. Once found, it can be extended, but efforts until then are little more than half-baked optimizations.

I think the problem with Kubernetes is that it demands a lot of cognitive load in the early stages of the project. There are a lot of things to think about at the beginning, which can be a hindrance to getting the project started and iterating quickly. Speed of feature development and speed of iteration are of Paramount importance in the early stages of a project. I think Heroku is an ideal development model for hosting a database using Postgres while deploying new code and serving it externally via Git push without much consideration. It might not scale indefinitely, and it might be expensive, but you don’t need to worry about that until your application gets to scale.

In short, Kubernetes makes simple things complicated and solving complex problems possible. If Kubernetes is to be truly successful, it needs to make the early stages of the project easier and reduce the cognitive burden on application developers. It’s not a problem to get developers to start learning the intricacies of Kubernetes at some point, because everything gets complicated and tricky with scale. However, as a successful development platform, these decision pressures should not be put on the developer’s face when they are not necessary. David Heinemeier Hansson (DHH), the father of Ruby on Rails, referred to this as “JIT Learning” in his RailsConf 2018 keynote.

I want to use Rails as an example: why was Rails so successful? Not because Ruby was popular (it was just a new programming language from Japan at the time), nor because Rails and Ruby were so good at dynamic web pages, but because they made developers more productive. DHH showed you how to create a blog in 15 minutes, which led to Java and. NET developers take note, because it takes them weeks or even months to build the same thing. Developers choose Rails because they can deliver functionality to users more quickly, which is exactly what these frameworks and infrastructures provide.

Rails creates the application skeleton for the developer and generates scaffolding and parts of the project code. Developers do not have to make a lot of decisions at the beginning of the project, such as how the application code should be organized, where the database model should be placed, what asset pipelines should be defined, how the database migration should take place, and other tasks and decisions.

I think Kubernetes could benefit from this kind of generator as well. There are already some specific resource generators, but they are far from sufficient. It would be nice if there were templates for different types of applications. The combination of relational databases, application layers, caching servers, message queues, and worker thread pools accounts for 90% or more of the applications built, and this simple structure can scale to incredible levels of complexity. Templates or generators that generate resources and code out of the box with Kubernetes organizational structure can be very useful.

A scaffold generator for generating common elements is a good one. Need a MySQL database? You can use commands like the following

kubectl scaffold mysql --generateCopy the code

To create a set of stateful services and other necessary things. Scaffolding can then be deployed into a K8S environment with a single command, and a few console commands can then be used to have a production-ready database. A similar approach can be used for popular application frameworks, message broker servers, and just about anything else we can think of.

Perhaps a combination of Operator and Helm would do the job, but I don’t think that’s good enough. There are two additional tools developers need to know about in addition to Kubernetes. Even learning simple technical terms and installing new command-line tools requires extra effort and thought. These things should be part of the Kubernetes out-of-the-box experience and can be accessed directly through Kubectl.

The growing number of projects in CNCF is not a particular problem for Kubernetes, but the CNCF project landscape is so large that the KubeCon/CNCF conference is very fragmented, with 14 topics in one session. As a developer, how do I know which tools are right for my project? Take a look at the projects in CNCF:

It is impossible to know all the items in the diagram. Application developers will have a much better experience if a pre-selected set of tools already exists. If they want to add different tools, that’s fine, but at least they shouldn’t have to think about it beforehand. CNCF’s growing complexity and wide reach may dilute Kubernetes’ branding as a building platform. I’m not sure what the outcome will be, and I’m not sure if I’m exaggerating, but it looks to me more like a hodgepodge of tools. When you spend a lifetime learning and building infrastructure tools, will you have the energy to solve users’ problems?

I want to emphasize that infrastructures exist to help application developers solve real-world problems for users, and they need to be constantly optimized to improve the efficiency and capability of delivery. Open platforms that can do this will win.

At some point, developers will need to learn more about infrastructure tools. Most developers working at AWS are familiar with some of AWS’s components, just as Google Cloud Platform or Azure developers are familiar with their clouds. Using Kubernetes as a base layer is more likely to succeed, because developers need only learn Kubernetes and can use it on any public or private cloud.

But Kubernetes’ learning curve has not been smooth. As an ecosystem, Kubernetes needs to meet the needs of application developers to be truly successful. After all, infrastructure is a supporting tool for solving user problems. Improving the productivity of application developers is the best way to ensure widespread adoption and success of a platform.

The original link: https://www.influxdata.com/blog/will-kubernetes-collapse-under-the-weight-of-its-complexity/