Author’s brief introduction
VIGNESH T.V. is the CEO, CTO and founder of Timecampus.
Kubernetes is currently the most popular open source container choreography platform, becoming the first choice for many enterprises to build their infrastructure. In this article, we’ll explore the best way to build the infrastructure for your use cases and the various decisions you might have to make based on the various constraints.
Architecture design
Your architecture should be designed largely around your use cases, so you need to be very careful during the design process to ensure that the infrastructure can support your use cases, and enlist the help of an external team of professionals if necessary. It’s important to be on the right track at the beginning of an architectural design, but that doesn’t mean there won’t be mistakes, and as new technology or research comes out every day, you can see that change becomes the norm and your architectural design thinking can become outdated.
This is why I strongly recommend that you adopt the principle of Architect for Chang and make your architecture a modular architecture so that you have the flexibility to change it internally in the future as needed.
Let’s look at how to achieve the goals of the system architecture with the client-server model in mind.
Entry point: DNS
In any typical infrastructure (cloud native or not), a message request must first be resolved by a DNS server and returned with the server’s IP address. Setting up your DNS should be based on the availability you need. If you need higher availability, you may want to spread your servers across multiple regions or cloud providers, depending on the level of availability you want to achieve.
Content Delivery Network (CDN)
In some cases, you may want to serve your users with as little latency as possible while reducing the load on your servers. This is where the Content Delivery Network (CDN) comes into play. Does the Client often request a set of static assets from the server? Do you want to speed up the delivery of content to users while reducing the load on your servers? In this case, using an edge CDN to service a set of static assets may actually help reduce user latency and server load.
Is all of your content dynamic? Can you provide delayed content to users to the extent that it reduces complexity? Or is your application receiving low traffic? In this case, it probably doesn’t make much sense to use a CDN, you can just send all the traffic directly to the global load balancer. But be aware that having a CDN also does have the advantage of distributing traffic, which can be very helpful if your server is under DDOS attack.
CDN providers include Cloudfare CDN, Fastly, Akamai CDN, Stackpath, and your cloud provider may also provide CDN services. For example, Cloud CDN of Google Cloud Platform, CloudFront of AWS, Azure CDN of Microsoft Azure, etc.
Load Balancer
If a request cannot be served by your CDN, the request is next sent to your load balancer. These can be regional IP addresses or global Anycast IP addresses. In some cases, you can also use a load balancer to manage internal traffic.
In addition to routing and proxying traffic to appropriate back-end services, load balancers can also assume responsibility for SSL termination, integration with CDN, and even managing certain aspects of network traffic.
While hardware load balancers exist, software load balancers offer great flexibility, reduced overhead, and elastic scalability.
Like CDN, your cloud provider should also be able to provide you with a load balancer (GLB for GCP, ELB for AWS, ALB for Azure, etc.), but more interestingly you can deploy these load balancers directly from Kubernetes. For example, creating an Ingress in GKE will also create a GLB at the back end for you to receive traffic. Other functions such as CDN and SSL redirection can also be set by configuring your Ingress. Visit the following link for details:
Cloud.google.com/kubernetes-…
Although you always start small, load balancers allow you to scale up to architectures with the following scales:
Network and security architecture
The next thing to watch is the web. If you want to improve security, you may need a private cluster. There, you can regulate inbound and outbound traffic, mask IP addresses behind NATs, isolate networks with multiple subnets over multiple VPCS, and more.
How you set up your network often depends on the degree of flexibility you seek and how you achieve it. Setting up the right network is about minimizing the attack surface while still maintaining normal operation.
Protecting your infrastructure by setting up the right network often also involves setting up firewalls with the right rules and restrictions to limit the flow of traffic in and out of various back-end services, both inbound and outbound.
In many cases, you can protect these private clusters by setting up a fortress host and tunneling all operations in the cluster, because what you need to expose to the public network is the fortress (aka Jump Host), usually set up on the same network as the cluster.
Some cloud providers also offer customized solutions for implementing zero-trust security. For example, GCP provides its users with identity awareness agents (Iaps) that can be used in place of a typical VPN implementation.
With all that taken care of, the next step is to set up the network within the cluster itself based on your use case. This involves the following tasks:
Setting up service discovery within the cluster (handled by CoreDNS)
Set up a service grid (e.g. LinkerD, Istio, Consul, etc.) if necessary
Set up Ingress Controller and API gateway (e.g. Nginx, Ambassador, Kong, Gloo, etc.)
Set up the network plug-in of CNI to facilitate the networking within the cluster
Set up network policies, regulate communication between services, and expose services using various service types as needed
Use protocols and tools such as GRPC, Thrift, or HTTP to set up inter-service communication between different services
Setting up A/B tests can be easier if you use A service grid like Istio or Linkerd
If you want to see some sample implementations, I suggest you take a look at the repo (github.com/terraform-g… Private Access, gKE-enabled shared VPCS, and more, all use Terraform. The interesting thing about the network in cloud computing is that it’s not limited to the cloud service provider in your area, but to multiple service providers across multiple regions as needed. This is where projects like Kubefed or Crossplane can help.
If you’d like to explore some of the best practices for setting up VPCS, subnets, and overall networks, I suggest you visit the page below. The same concepts apply to any cloud provider you join:
Cloud.google.com/solutions/b…
Kubernetes
If you are using a managed cluster like GKE, EKS, AKS, Kubernetes is automatically managed to reduce the complexity of user operations.
If you manage Kubernetes yourself, you need to deal with a lot of things, such as backing up and encrypting etCD storage, networking between nodes in the cluster, regularly patching your nodes with the latest version of the operating system, and managing cluster upgrades to keep up with upstream versions of Kubernetes. For this reason, this is only recommended if you have a dedicated team to maintain these things.
Site Reliability Engineering (SRE)
When you’re maintaining a complex infrastructure, it’s important to have an observable stack in place so that you can detect errors and anticipate possible changes before users notice them, identify anomalies, and have the freedom to drill down to see what the problem really is.
Now, this requires that you have proxies that expose metrics to specific tools or applications to collect and analyze (either by following pull or push mechanisms). If you are using a service grid with Sidecars, they tend to come with their own metrics and don’t require custom configuration.
In any scenario, you can use a tool like Prometheus as a timing database to collect all metrics for you, and with a tool like OpenTelemetry, expose metrics from applications and various tools with built-in Exporter. With tools like Alertmanager to send notifications and alerts to multiple channels, Grafana will provide visual dashboards that give users complete visibility into their entire infrastructure.
Taken together, this is Prometheus’s observable solution:
With such a complex system, you also need to use a log aggregation system so that all logs can flow into one place for easy debugging. Most organizations tend to use ELK or EFK stacks, Logstash or FluentD to aggregate and filter logs for you based on your limitations. But there are new players in the logging space, like Loki and Promtail.
The following diagram illustrates how a log aggregation system like FluentD can simplify your architecture:
But what if you want to track requests across multiple microservices and tools? This is where distributed tracing comes into play, especially given the complexity of microservices. Tools like Zipkin and Jaeger have been pioneers in this space, and the latest newcomer to this space is Tempo.
While log aggregation gives information from a variety of sources, it doesn’t necessarily give the context of the request, which is where tracing really helps. But keep in mind that adding a trace to your stack can add a lot of overhead to your request, because the context must be propagated between services along with the request.
Here is a typical distributed trace architecture:
However, website reliability is not just about monitoring, visualization, and alerting. You must be prepared to deal with any failure in any part of your system and do regular backups and failover to at least minimize data loss. You can do this with tools like Velero.
Velero helps you maintain regular backups of various components in your cluster, including your workload, storage, and more, by leveraging the same Kubernetes architecture you use. Velero’s architecture is as follows:
saved
There are many different stored programs and file systems available, which can vary widely between cloud providers. This requires a standard like the Container Storage Interface (CSI), which helps with external plug-ins for most volumes so that they are easy to maintain and grow without becoming a core bottleneck.
The CSI architecture is shown below, which typically supports a variety of volume plug-ins:
What about clustering, expansion and other problems caused by distributed storage? At this point file systems like Ceph have proven themselves, but given that Ceph is not built around Kubernetes and is somewhat difficult to deploy and manage, consider a project like Rook.
Although Rook is not coupled to Ceph and supports other file systems such as EdgeFS, NFS, etc., Rook and Ceph CSI seem like a natural fit. The structure of Rook and Ceph is as follows:
As you can see, Rook takes over the installation, configuration, and management of Ceph in the Kubernetes cluster. The following storage is automatically allocated according to the user’s preference. All of this happens without exposing the application to any complexity.
Mirror warehouse
The mirror repository provides you with a user interface where you can manage various user accounts, push/pull images, manage quotas, get event notifications via Webhook, scan for vulnerabilities, sign pushed images, process images or copy images across multiple mirror repositories, and so on.
If you are using a cloud provider, chances are they already provide a mirror warehouse as a service (such as GCR, ECR, ACR, etc.), which removes a lot of the complexity. If your cloud provider doesn’t offer one, you can also opt for third-party repositories like Docker Hub, Quay, etc.
But what if you want to host your own mirror warehouse?
Hosting may be required if you want to deploy a mirror repository in-house, have more control over itself, or reduce the costs associated with operations such as vulnerability scanning.
If this is the case, choosing a private mirror repository like Harbor can help. Harbor architecture is as follows:
Harbor is an OCI-compliant mirror repository consisting of various open source components including Docker Mirror Repository V2, Harbor UI, Clair, and Notary.
CI/CD architecture
Kubernetes can host all workloads on any scale, but this also requires a standard way to deploy applications with a streamlined CI/CD workflow. The following figure shows a typical CI/CD pipeline:
Now, although the steps generally remain the same if you manage your own CI runner, you need to configure them to be set inside or outside the cluster and have the appropriate permissions to push assets to the mirror repository.
Total knot
We have introduced the kubernetes-based cloud native infrastructure architecture. As we’ve seen above, various tools address different infrastructure issues. Like Lego bricks, they each focus on a specific problem of the moment, abstracting a lot of complexity for you.
This allows users to get used to Kubernetes in an incremental way. And you can use only the tools you need in the entire stack, depending on your use case.