1 What is LXCFS?

LXCFS is a small FUSE file system designed to make the Linux container more like a virtual machine. LXCFS looks at the files provided under procfs, such as:

/proc/cpuinfo
/proc/diskstats
/proc/meminfo
/proc/stat
/proc/swaps
/proc/uptime
/proc/slabinfo
/sys/devices/system/cpu/online
Copy the code

2 Why LXCFS?

Linuxs uses Cgroup to implement resource restriction on containers, but still mounts the /proc directory of procfs on the host by default inside the container, which contains resource information such as meminfo, cpuinfo, stat, uptime and so on. Monitoring tools such as Free/Top or legacy applications also rely on the above file content for resource configuration and usage. When they run in a container, they can read the state of the host’s resources, causing errors and inconvenience.

3 How to apply LXCFS?

3.1 the docker practice

  • The installation
yum install fuse fuse-lib fuse-devel
git clone git://github.com/lxc/lxcfs
cd lxcfs
./bootstrap.sh
./configure
make
make install
Copy the code
  • Write service startup scripts
mkdir -p /var/lib/lxcfs
cat > /usr/lib/systemd/system/lxcfs.service <<EOF
[Unit]
Description=lxcfs

[Service]
ExecStart=/usr/bin/lxcfs -f /var/lib/lxcfs 
Restart=on-failure
#ExecReload=/bin/kill -s SIGHUP $MAINPID

[Install]
WantedBy=multi-user.target
EOF
Copy the code
  • Start the LXCFS
systemctl daemon-reload
systemctl start lxcfs
Copy the code
  • Start the container and view
docker run -it -m 256m \
      -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw \
      -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw \
      -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw \
      -v /var/lib/lxcfs/proc/stat:/proc/stat:rw \
      -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw \
      -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw \
      ubuntu:latest /bin/bash
root@e203bbf2c394:/# free
              total        used        free      shared  buff/cache   available
Mem:         262144        1124      261016           0           4      261020
Swap:        524288           0      524288
Copy the code

3.2 kubernetes practice

  • validation
# see whether open $kubectl API - versions | grep 'admissionregistration. K8s. IO/v1beta1 admissionregistration. K8s. IO/v1beta1Copy the code
  • Download the LXCFS – Admission – Webhook program
$ git clone https://github.com/denverdino/lxcfs-admission-webhook.git
$ cd lxcfs-admission-webhook
Copy the code
  • Deploy daemonset – LXCFS
$ kubectl apply -f deployment/lxcfs-daemonset.yaml
$ kubectl  get pod | grep lxcfs
lxcfs-ch95x                                          1/1     Running   0          2d22h
lxcfs-j6pjk                                          1/1     Running   0          2d22h
Copy the code
  • Deploy LXCFS – admission – webhook
$# Execute shell deployment script $deployment/install.shCopy the code
  • test
$ kubectl label namespace default lxcfs-admission-webhook=enabled $ kubectl apply -f deployment/web.yaml $ kubectl get pod | grep web web-79c99559c7-fmqhl 1/1 Running 0 23h web-79c99559c7-mlzkb 1/1 Running 0 23h $ kubectl exec -it Web-79c99559c7-fmqhl -- free-h total Used free shared buffers cached Mem: 256M 2.6m 0B 0B 272K -/+ buffers/cache: 2.4m 253M Swap: 0B 0B 0BCopy the code

Q&A

The following error occurs when Kubernetes deploys Daemonset:

kubectl  logs -f lxcfs-rlnl9
/usr/local/bin/lxcfs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory
Copy the code

The solution

  • Methods a

Install fuse-libs on each node

yum install fuse-libs -y
Copy the code
  • Method 2

Change the deployment script and repackage. It has been submitted to Github PR as PR and has not been merged yet

  • lxcfs-image/Dockerfile
FROM centos:7 as build RUN yum -y update RUN yum -y install fuse-devel pam-devel wget install gcc automake autoconf Libtool make ENV LXCFS_VERSION 3.1.2 wget RUN https://linuxcontainers.org/downloads/lxcfs/lxcfs-$LXCFS_VERSION.tar.gz &&  \ mkdir /lxcfs && tar xzvf lxcfs-$LXCFS_VERSION.tar.gz -C /lxcfs --strip-components=1 && \ cd /lxcfs && ./configure && make FROM centos:7 STOPSIGNAL SIGINT COPY --from=build /lxcfs/lxcfs /usr/local/bin/lxcfs COPY --from=build /lxcfs/.libs/liblxcfs.so /usr/local/lib/lxcfs/liblxcfs.so COPY --from=build /lxcfs/lxcfs /lxcfs/lxcfs COPY --from=build / LXCFS/libs/liblxcfs. So/LXCFS/liblxcfs. So the COPY - from = build/usr/lib64 / libfuse. So. 2.9.2 / LXCFS/libfuse. So. 2.9.2 COPY - the from = build/usr/lib64 / libulockmgr. So. / LXCFS libulockmgr 1.0.1. So. COPY start 1.0.1. Sh/CMD / / start. Sh"Copy the code
  • lxcfs-image/start.sh
#! /bin/bash # Cleanup nsenter -m/proc/1/ns/mnt fusermount -u /var/lib/lxcfs 2> /dev/null || true nsenter -m/proc/1/ns/mnt [ -L /etc/mtab ] || \ sed -i "/^lxcfs \/var\/lib\/lxcfs fuse.lxcfs/d" /etc/mtab # Prepare mkdir -p /usr/local/lib/lxcfs /var/lib/lxcfs # Update lxcfs cp -f /lxcfs/lxcfs /usr/local/bin/lxcfs cp -f /lxcfs/liblxcfs.so / usr/local/lib/LXCFS/liblxcfs so cp - f/LXCFS libfuse. So the 2.9.2 / usr/lib64 / libfuse. So. 2.9.2 cp - f / LXCFS libulockmgr. So. / usr/lib64 / libulockmgr 1.0.1. So. The 1.0.1 ln -s/usr/lib64 / libfuse. So the 2.9.2 / usr/lib64 / libfuse. So. 2 ln - s/usr/lib64 / libulockmgr. So. / usr/lib64 / libulockmgr 1.0.1. So. 1 # Mount exec nsenter -m/proc / 1 / ns/MNT /usr/local/bin/lxcfs /var/lib/lxcfs/Copy the code
  • lxcfs-image/Dockerfile
apiVersion: apps/v1 kind: DaemonSet metadata: name: lxcfs labels: app: lxcfs spec: selector: matchLabels: app: lxcfs template: metadata: labels: app: lxcfs spec: hostPID: true tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule containers: - name: lxcfs image: Registry.cn-hangzhou.aliyuncs.com/denverdino/lxcfs:3.1.2 imagePullPolicy: Always securityContext: ring: true volumeMounts: - name: cgroup mountPath: /sys/fs/cgroup - name: lxcfs mountPath: /var/lib/lxcfs mountPropagation: Bidirectional - name: usr-local mountPath: /usr/local - name: usr-lib mountPath: /usr/lib64 volumes: - name: cgroup hostPath: path: /sys/fs/cgroup - name: usr-local hostPath: path: /usr/local - name: usr-lib hostPath: path: /usr/lib64 - name: lxcfs hostPath: path: /var/lib/lxcfs type: DirectoryOrCreateCopy the code