The test environment

  1. Ubuntu 21.10 (kernel 5.13.0-37 – generic)
  2. Docker 20.10.14
  3. Kubectl 1.23.5
  4. Minikube 1.25.2
  5. Rook 1.8.7

The preparatory work

Downloading the System Image

This installation uses the Ubuntu Server version. Download mirror can go directly to the domestic mirror stands for, Ubuntu 21.10 version of the code for the impish, for example, you can go to mirrors.tuna.tsinghua.edu.cn/ubuntu-cdim… Find the corresponding version and select the ISO file to download.

Prepare a good network environment

Because Kubernetes needs to go to k8s.gcr. IO warehouse to pull the image, the way of label replacement has not found a good automation scheme, so it is more convenient to prepare a network agent.

Vm Software

Rook officially recommends setting up tests in a virtual machine environment. Improper test operation may cause the file system of the device to be overwritten!

Install the system

Ubuntu configuration is as follows:

  1. CPU: 2 cores (Minikube minimum 2 cores)
  2. Memory: 4G (Minikube minimum 2G MEMORY)
  3. Hard disk: 50GB (for system installation, customizable)
  4. Add three virtual hard disks for creating OSD nodes and set them to 10 GB

The VM network will adopt NAT mode, choose to install Openssh Server during installation, not at this time to install Docker (otherwise, snap will be used to install Docker, the next configuration is inconsistent with the tutorial).

Install the Docker

Sudo apt Upgrade sudo apt Upgrade sudo apt upgrade sudo apt upgrade

According to the official docker tutorial, execute commands to install the required parts, but Ubuntu21.10 has been installed so you don’t need to perform this step.

sudo apt-get update
sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
Copy the code

Download Docker official GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
Copy the code

Add to the local configuration file

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Copy the code

Install the Docker

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io
Copy the code

Add the local user to the Docker group for the convenience of using the Docker command, and log out the shell once after setting

sudo usermod -aG docker $USER && newgrp docker
Copy the code

Configure the network

First configure the Docker network Settings

sudo mkdir -p /etc/systemd/system/docker.service.d
Copy the code

Create the/etc/systemd/system/docker. Service. D/HTTP – proxy. Conf and write the following content

[Service] Environment="HTTP_PROXY=http://proxy.example.com:80" Environment="HTTPS_PROXY=https://proxy.example.com:443" The Environment = "NO_PROXY = localhost, 127.0.0.1, 192.168.49.0/24, NAT IP segment > < virtual machine / 24"Copy the code

For example, if the VM IP address is 192.168.104.x, enter 192.168.104.0/24

And save a copy of the shell Settings for Kubernetes to use

export HTTP_PROXY=http://proxy.example.com:80 export HTTPS_PROXY=https://proxy.example.com:443 export NO_PROXY = localhost, 127.0.0.1, 192.168.49.0/24, NAT IP segment > < virtual machine / 24Copy the code

Since pull mirrors are usually transferred over HTTPS, it is recommended that HTTP_PROXY be removed

Install kubectl

No agent is required for this step

In the Linux system installation and set the kubectl | Kubernetes

You are advised to install the plug-in to improve efficiency

Install minikube

Minikube official installation tutorial

Note that the agent needs to be configured after minikube start

Install the Rook

After configuring the proxy, run minikube start to open Kubernetes.

Clone rook warehouse

Git clone, single - branch, branch v1.8.7 https://github.com/rook/rook.gitCopy the code

Create a rook – ceph – operator

cd rook/deploy/examples kubectl create -f crds.yaml -f common.yaml -f operator.yaml # Kubectl -n rook-ceph get pod kubectl -n rook-ceph get pod kubectl -n rook-ceph get podCopy the code

Due to the use of the Minikube, so choose the rook/deploy/examples/cluster – test. Yaml files to create cluster

Remove the deviceFilter comment from cluster-test.yaml

Virtual machine hard disk/dev/sda, respectively, / dev/SDB, / dev/SDC, / dev/SDD, with sda for file system to the system, so need to fill in in deviceFilter sd [^ a], located on the sda to ignore create the osd.

run

kubectl create -f cluster-test.yaml
Copy the code

After creating the cluster, ensure that the Pods contain rook-ceph-mon, rook-ceph-Mgr, and three rook-ceph-OSD, create rook-Ceph-Toolbox to check the ceph status.

The cepH status should be HEALTH_OK.

Deploying object Storage

Because erasureCoded Chunk requires at least three OSD nodes, three hard disks or three unformatted partitions are required. Since the cluster has only one node, you need to set the failureDomain to OSD rather than host.

run

kubectl create -f object.yaml
Copy the code

The RGW is created and the deployment is complete.