“This is the second day of my participation in the November Gwen Challenge. See details of the event: The last Gwen Challenge 2021”.

Hello everyone, I’m Zhang Jintao.

In this article, I will introduce OCI and Docker image related content, welcome to comment and discuss.

OCI’s past lives

DotCloud first demonstrated Docker at PyCon in March 2013, and later declared it open source. Docker began to be known, and a wave of containerization followed.

Docker 1.0 was officially released in June 2014 with nearly 460 contributors and over 8,700 submissions, marking Docker’s production-ready status.

At the time, the first thing to think about containerization was Docker. At that time, the implementation or development direction of Docker was mainly determined by Docker Inc. There is no single industry standard that companies control. This is obviously unacceptable for some head companies. There is no unified industrial standard, which means that if Docker container technology is chosen, Docker Inc. Bound to the company; In addition, with the upgrade of Docker software, certain functions or features are bound to change, and no one can guarantee against destructive changes.

Therefore, in order to promote the industrial standardization of containerization technology, the Linux Foundation and Google, Huawei, HP, IBM, Docker, Red Hat, VMware and other companies jointly announced the establishment of open Container Project (OCP) at DockerCon in June 2015. It was renamed OCI. Its primary goal is to establish industry-wide open standards for container formats and runtimes.

Up to now, OCI has developed three major standards, namely run-time spec, image-spec, and Distribution-spec, which define the container runtime, container image, and distribution specifications respectively, which will be described later.

In order to support the advancement of OCI container runtime standard, Docker company drafted the draft of image format and runtime specification, and donated the relevant implementation of Docker project to OCI as the basic implementation of container runtime, now named RUNC.

Docker later turned its container runtime into a separate project called Containerd and donated it to CNCF, which is now a graduate project of CNCF.

OCI image vs Docker image

The establishment of OCI has promoted industrial standardization of container technology, but is this standard the only one? It’s not. When OCI was established and image-Spec standards were formulated, Docker was already prosperous and widely used.

Since the standard only defines the most basic content, it will cause destructive changes to Docker and is not conducive to the iteration of Docker functions if Docker implementation is reformed in accordance with the standard.

Therefore, in order to support the popularization of OCI standard, Docker has promoted registry’s support for OCI images. Now, it is also adding adaptation to Docker itself. The goal is to make Docker support two image formats. Docker-compliant images and OCI-compliant images respectively.

So what are the similarities and differences? Let’s look at it step by step.

The difference and connection between Docker Image and OCI Image

In my previous articles, we have introduced in detail what a Docker image is fundamentally. Here we will give a quick introduction.

Each Docker image is organized by a series of configuration lists and corresponding layers. Each layer is typically an archive in tar format, and the configuration manifest describes the order in which the corresponding layer should be organized, along with some meta attributes of the image. For example, the architecture supported by the image, such as AMD64, and some pre-configured parameters such as ENV, etc.

Of course, the Docker Image also contains the Docker version used in Image construction, docker_version, and the history of Image construction history and other information. So you can see the version of Docker used to build the IMAGE on DockerHub or other IMAGE repositories, or you can view the build history by using Docker history .

So what is an OCI Image? First of all, we need an OCI Image to explore what it is.

Here is a tool called Skopeo that can easily convert an OCI Image from an Image repository, a local Docker daemon, or even a Docker Image tar file saved by Docker Save.

The skopeo installation process will not be described, but refer to the documentation on the project home page. So let’s go ahead and use it.

Let’s use a debian mirror as an example.

(MoeLove) ➜ Skopeo copy docker:// Debian: Stretch OCI: Debian: Stretch Getting Image source signatures Copying blob a4d8138d0f6b done Copying config 45f82268e3 done Writing manifest to image destination Storing signaturesCopy the code

The OCI Image is created by using the command above. Let’s look at its directory structure.

(MoeLove) ➜ Tree ├── ├─ sha256 │ ├─ 0043 cd2a654fe86258f43f5b1dbbb4e6c582cc4bb6e505e9c5171c124150d155 │ ├ ─ ─ 45 f82268e32180cb1839f90467d9b8a8258953d68b7221199976653308d92ef5 │ └ ─ ─ A4d8138d0f6b5a441aaa533faf5fe0c3996a6ca42643c46f4402c7e8bda53742 ├ ─ ─ index. Json └ ─ ─ oci - layout 2 directories and filesCopy the code

Does it feel like deja vu? It is true that the specification of OCI Image is established on the basis of Docker Image, so it generally looks not very different. Let’s look at what’s going on.

oci-layout

This file is the layout file for the OCI Image and is used to describe the Image specification it uses or follows.

(MoeLove) ➜ debian cat oci - layout | jq {" imageLayoutVersion ":" 1.0.0 "}Copy the code

You can see that the content here says 1.0.0, which indicates that the image follows the OCI version 1.0.0 layout specification.

index.json

The manifest field in the index.json file is similar to the manifest.json in Docker Image. As the top-level configuration of OCI Image, it is also an entry configuration of the Image.

(MoeLove) ➜ debian cat index. Json | jq {" schemaVersion ": 2," manifests ": [{" mediaType" : "application/vnd.oci.image.manifest.v1+json", "digest": "sha256:0043cd2a654fe86258f43f5b1dbbb4e6c582cc4bb6e505e9c5171c124150d155", "size": 349, "annotations": { "org.opencontainers.image.ref.name": "stretch" }, "platform": { "architecture": "amd64", "os": "linux" } } ] }Copy the code

From its contents, you can see that the mediaType fields in it have the same form as the types in the Docker Image, but the Docker has been replaced with OCI. From this configuration file, we can find the first blob is sha256:0043 cd2a654fe86258f43f5b1dbbb4e6c582cc4bb6e505e9c5171c124150d155 we look at the content.

(MoeLove) ➜ debian cat blobs/sha256/0043 cd2a654fe86258f43f5b1dbbb4e6c582cc4bb6e505e9c5171c124150d155 | jq { "schemaVersion": 2, "config": { "mediaType": "application/vnd.oci.image.config.v1+json", "digest": "sha256:45f82268e32180cb1839f90467d9b8a8258953d68b7221199976653308d92ef5", "size": 579 }, "layers": [ { "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", "digest": "sha256:a4d8138d0f6b5a441aaa533faf5fe0c3996a6ca42643c46f4402c7e8bda53742", "size": 45337510 } ] }Copy the code

This entry file describes the actual configuration of the OCI image and the Layer configuration within it. If there are multiple layers then layers will increase accordingly.

Note: The layers of mediaType USES the application/VND. Oci. Image. Layer. V1. Tar + gzip data content is after gzip compression If you have interest you can use it the tar decompressed, you will find interesting content.

I’m just going to say the result here, and when you unpack it you’ll get a rootFS which is similar to a Docker Image.

summary

We obtained OCI Image from debian Docker Image from local Docker Daemon through Skopeo tool, and analyzed its content.

The main differences are that the directory structure is not exactly the same, and the configuration information, especially the specification of mediaType, is different.

Their connection also lies in this. The specification of OCI Image is modified from the specification of Docker Image, so the organization form of bloBs similar to them is roughly the same, and many parameters in the configuration file are similar.

In addition, we can easily get another conclusion, that is, we can easily convert Docker Image into OCI Image.

OCI image and Docker image conversion

As we have seen above, skopeo can convert a Docker Image to an OCI Image, and of course it can also convert an OCI Image to a Docker Image. Here is the method:

➜ skopeo copy Docker :// Debian :stretch oci:debian:stretch Getting image source signatures Copying blob a4d8138d0f6b done Copying config 45f82268e3 done Writing Manifest to image destination signatures # Convert the debian OCI image in the current directory to a Docker image and store it in the local Docker daemon (MoeLove) ➜ Skopeo Copy OCI: Debian: Stretch Docker-Daemon :local/ Debian: OCI Getting Image source signatures Copying blob 0e350e141713 Done Copying config aAE58A37CF done Writing manifest to image destination Storing signatures # verification (MoeLove) ➜ OCI docker images local/ Debian REPOSITORY TAG IMAGE ID CREATED SIZE local/ Debian OCI AC6bCF605d82 6 months ago 101MBCopy the code

Image builder

In CI/CD environment, we can use DinD (Docker in Docker) to start a Docker daemon or mount the external /var/run/docker.sock to the container. Or expose the way the Docker API uses HTTP and build directly from that address.

But do these ways make you feel heavier? Are there safety concerns, or pressure and load concerns?

The pressure and load here mainly refer to the pressure caused to the Same Docker daemon when all tasks share the same service.

Here we introduce some other image building tools that allow you to build images without Docker and upload them to the Docker image repository.

So far, we have had a number of options:

  • BuildKit
  • img
  • orca-build
  • umoci
  • buildah
  • kaniko
  • FTL
  • Bazel rules_docker

These tools have different focuses, and there are not only the tools listed above, but these tools are typical.

Buildah is usually advertised as the next generation of image building tools on the web, mostly because it can build OCI or Docker images directly, as well as Dockerfile directly. And it can also pull/push images, can be said to be fully compatible with Docker in image building, can even be said to be a Docker replacement in image building.

Buildah does not require any root permissions to build images, does not rely on Docker, uses a simple fork-exec model, and can also be included as a library in other tools. The ultimate goal is to provide a lower-level core set of tools to do things related to building mirrors.

With that typical substitution out of the way, we’ll talk about BuildKit and IMG. Img is built on top of BuildKit, so there are a lot of similarities. They use non-root users to build images. Of course, BuildKit, which I covered in detail in a previous article, is a next-generation build tool built into Docker and can be used independently. It’s fair to call it “the next generation of image building tools.”

Kaniko, a Google product whose main slogan is “Build container images in Kubernetes,” actually works in both K8S clusters and containers. It can also build images using Dockerfile. Of course, it is also very important that all of its build commands are run in user mode, and can be well combined with Kubernetes, in the cloud native era, it also has a certain advantage.

The above tools are only briefly introduced, but if you are interested in them, you can go to the project home page to view the readme. md tools.

conclusion

This article introduces the past and present of OCI, as well as the specifications and characteristics of OCI Image, and also introduces a tool skopeo that can be used for Image conversion between OCI Image and Docker Image. In addition, some tools that can be used to replace Docker build in CI environment or other environment with specific scenarios are introduced. Please choose according to your actual needs.

Please feel free to subscribe to my official account [MoeLove]