Embedded Linux container technology

First, Linux container technology

Linux Container is a kernel virtualization technology that provides lightweight virtualization to isolate processes and resources.
LXC is short for Linux Container. Lightweight virtualization can be provided to isolate processes and resources without the need to provide instruction interpretation mechanisms and other complexities of full virtualization. Equivalent to NameSpace in C++. Containers effectively divide resources managed by a single operating system into isolated groups to better balance conflicting resource usage requirements among isolated groups. Compared with traditional virtualization technologies, it has the following advantages:

(1) Using the same kernel with the host, the performance loss is small;

(2) no instruction level simulation is required;

(3) No just-in-time compilation is required;

(4) The container can run instructions locally in the CPU core without any special interpretation mechanism;

(5) Avoid the complexity of paravirtualization and system call replacement;

(6) Lightweight isolation. In isolation, it also provides a sharing mechanism to realize resource sharing between the container and the host.

Bottom line: Linux Container is a lightweight virtualization tool.
Linux Container supports the simultaneous execution of multiple isolated Server Containers on a single controllable host node. Linux Container is a bit like chroot, providing a virtual environment with its own processes and network space, but it is different from a virtual machine because LXC is a virtualization of resources at the operating system level.
Relationship between LXC and Docker

(1) Docker is not a substitute for LXC. The bottom layer of Docker is implemented by LXC. LXC sandboxes Linux processes, isolating them from each other, and controlling resource allocation for each process.

(2) On the basis of LXC, Docker provides a series of stronger functions.

Embedded Linux container

Mainstream Linux containers include LXC and Docker, among which LXC is the native lightweight container technology of Linux. Docker is implemented based on LXC. However, the main application scenario of Docker is cloud server, which has very powerful functions. However, its support for embedded Linux development platform may not be so friendly. BalenaOS claims to streamline Docker so it can run reliably in embedded Linux environments. It is based on the Yocto framework, and Systemd as init system. In comparison, LXC is lightweight and requires less system resources, so it is a natural fit for embedded Linux environments. However, (balenaOS) Docker and LXC each have their strengths and weaknesses, and it takes a deep experience to reach a final conclusion.

3. Experience container

3.1 the LXC

3.1.1 Hardware Environment

allwinner T3
Four nuclear architecture (A7
1G RAM
Ubuntu Core 18.04 (Linux Kernel 3.10.108)

3.1.2 LXC installation

System requirements:

Mandatory requirements:
1. A C library in glibc, LIbc, or UClib
2. The Linux kernel > = 2.6.32
Lxc-attach Additional requirements:
1. The Linux kernel > = 3.8
Additional requirements for non-privileged containers:
1. Libpam-cgfs, CGManager, or another CGroup manager configures the system for non-privileged CGroups operations
2. The latest version of Shadow, including NewuidMap and Newgidmap
3. The Linux kernel > = 3.12
Recommend a library:
1. libcap (to allow for capability drops)
2. libapparmor (to set a different apparmor profile for the container)
3. libselinux (to set a different selinux context for the container)
4. libseccomp (to set a seccomp policy for the container)
5. libgnutls (for various checksumming)
6. liblua (for the LUA binding)
7. python3-dev (for the python3 binding)

LXC installation in Ubuntu is very simple. It takes only one command to install LXC, including all the library files it depends on:

	$sudo  apt-get  install  lxc
Copy the code

3.1.3 Creating a non-privileged Container

An unprivileged container is the most secure container. Those that use the mapping of uids and Gids to assign a set of Uids and Gids to the container. This means that UID 0 (root) in the container is actually like UID 100000 outside the container. Therefore, if something goes wrong and an attacker manages to escape the container, they will find that they have just as much power as an empty user.

Unfortunately, this also means that the following common operations are not allowed:

Install most file systems
Creating a Device Node
Any operations on uids/Gids outside of the mapped set

Note: Because the kernel version of the Ubuntu Core environment in this test is 3.10.108, the system does not support unprivileged containers.

3.1.4 Creating a privilege container

** Note :** The following created containers are privileged containers, that is, containers are created and started by root.

After LXC is installed, you can use lxC-checkConfig to check whether the Linux kernel supports containers.

	$ lxc-checkconfig

	--- Namespaces ---
	Namespaces: enabled
	Utsname namespace: enabled
	Ipc namespace: enabled
	Pid namespace: enabled
	**User namespace: missing**
	Network namespace: enabled
	Multiple /dev/pts instances: enabled
	
	--- Control groups ---
	Cgroups: enabled
	
	Cgroup v1 mount points: 
	/sys/fs/cgroup/systemd
	/sys/fs/cgroup/blkio
	/sys/fs/cgroup/cpuset
	/sys/fs/cgroup/devices
	/sys/fs/cgroup/perf_event
	/sys/fs/cgroup/cpu,cpuacct
	/sys/fs/cgroup/memory
	/sys/fs/cgroup/freezer
	
	Cgroup v2 mount points: 
	
	
	Cgroup v1 clone_children flag: enabled
	Cgroup device: enabled
	Cgroup sched: enabled
	Cgroup cpu account: enabled
	Cgroup memory controller: enabled
	Cgroup cpuset: enabled
	
	--- Misc ---
	Veth pair device: enabled, loaded
	Macvlan: enabled, not loaded
	Vlan: enabled, not loaded
	Bridges: enabled, loaded
	Advanced netfilter: enabled, not loaded
	CONFIG_NF_NAT_IPV4: enabled, loaded
	CONFIG_NF_NAT_IPV6: enabled, not loaded
	CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
	CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
	CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, loaded
	CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded
	FUSE (for use with lxcfs): enabled, not loaded
	
	--- Checkpoint/Restore ---
	checkpoint restore: enabled
	CONFIG_FHANDLE: enabled
	CONFIG_EVENTFD: enabled
	CONFIG_EPOLL: enabled
	CONFIG_UNIX_DIAG: enabled
	CONFIG_INET_DIAG: enabled
	CONFIG_PACKET_DIAG: enabled
	CONFIG_NETLINK_DIAG: enabled
	File capabilities: 
	
	Note : Before booting a new kernel, you can check its configuration
	usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig
Copy the code

This command checks whether the kernel supports containers based on the kernel’s.config file.

LXC – create to create a container that LXC, when creating a container provides many template, this template location: / usr/share/LXC/templates/directory.

	sudo lxc-busybox  lxc-download  lxc-local  lxc-oci
Copy the code

Lxc-download create a container with distribution to Ubuntu, release to Bionic and arch to ARMHF:

	sudo lxc-create -n c1 -t download
Copy the code

The above command is used to create a container named c1. After typing this command, it will prompt for system, system version, architecture and other parameters, and then LXC will download the corresponding container rootfs. Rootfs may take a long time to download. If rootfs is successfully downloaded, the container has been successfully downloaded. The rootfs directory is located in the /var/lib/lxc/c1 directory, which contains a config configuration file and the rootfs system.

Through lxc-ls, you can see that C1 container has been deployed in the system, and lxc-info, you can view the current running status of C1.

The lxc-start command is used to start a container. You can run the lxc-start -h command to view specific parameters.

    sudo lxc-start -n c1 -o logfile -d
Copy the code

The problem

1. Apparmor configuration problem

1 lXC-start C1 20190812055509.777 ERROR apparmor-lSM /apparmor.c:apparmor_process_label_set: 223-if you really want to  start this container, Set 2 lxC-start C1 20190812055509.812 ERROR apparmor-lSM /apparmor.c:apparmor_process_label_set:224 -lxc. Apparmor. Allow_incomplete = 1 3 lxc-start c1 20190812055509.822 ERROR apparmor - LSM /apparmor. C :apparmor_process_label_set: 225-in your Container Configuration file 4 lXC-start C1 20190812055509.842 ERROR sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 5) 5 lxc-start c1 20190812055509.908 ERROR lXCContainer -LXCContainer. c: wait_ON_DAemonized_start: 842-received Container state "ABORTING" instead of "RUNNING" 6 lxc-start c1 20190812055509.953 ERROR lxc_start-tools /lxc_start.c:main:330 - The Container failed to start 7 lxc-start c1 20190812055509.970 ERROR lxc_start-tools /lxc_start.c:main:333 - to get more details, Run the container in foreground mode 8 lxc-start c1 20190812055509.984 ERROR lxc_start-tools/lxc_start_c :main:336 - Additional information can be obtained by setting the --logfile and --logpriority options 9 lxc-start c1 20190812055509.577 ERROR start-start. c:__lxc_start:1939 - Failed to spawn container "c1"Copy the code

By analyzing the logs, we can see that “lxc.apparmor. Allow_incomplete = 1” needs to be added to the configuration file of the container. The c1 container configuration file is /var/lib/lxc/c1/config. Add this configuration to the end of the config.

After the above steps are completed, the container C1 is started again, but it is still not started successfully, and the error log is:

1 lxc-start c1 20190812034836.659 ERROR lSM-LSM/lSM. c: lSM_process_label_set_at :174 - No such file or directory - Failed to set AppArmor label "lXC-container-default" 2 lXC-start c1 20190812034836.659 ERROR AppArmor - lsm/apparmor.c:apparmor_process_label_set:245 - Failed to change apparmor profile to lxc-container-default 3 lxc-start C1 20190812034836.660 ERROR sync-sync. C :__sync_wait: 62-an ERROR occurred in another process (expected sequence Number 5) 4 LXC -start C1 20190812034836.660 ERROR lXCContainer -lXCcontainer. c: wait_on_DAemonized_start: 842-received Container state "ABORTING" instead of "RUNNING" 5 lxc-start c1 20190812034836.661 ERROR lxc_start - C :main:330 - The container failed to start 6 lxc-start c1 20190812034836.662 ERROR lxc_start - tools/lxc_start.c:main:333 - To get more details, Run the container in foreground mode 7 lxc-start c1 20190812034836.662 ERROR lxc_start-tools /lxc_start.c:main:336 - Additional information can be obtained by setting the --logfile and --logpriority options 8 lxc-start c1 20190812034836.719 ERROR start-start. c:__lxc_start:1939 - Failed to spawn container "c1"Copy the code

Analyzing logs, AppArmor could not find the “lxc-container-default” configuration file. AppArmor is a selinux-like security module provided by the Linux kernel and supported by Ubuntu by default. However, By default, Ubuntu Core does not support apparmor_status and apparmor_parser tools, and the system has few built-in profiles. Can be installed via sudo apt-get install apparmor-profiles, above apparmor_status, apparmor_parser tools will also be installed.

After installing the apparmor profile, it is necessary to restart apparmor as follows:

	Start : sudo /etc/init.d/apparmor start
	Stop : sudo /etc/init.d/apparmor stop
	reload: sudo /etc/init.d/apparmor reload
	Show status: sudo /etc/init.d/apparmor status
Copy the code

Run sudo lxc-info -n c1 to view the running status of the container:

Sudo lxc-info -n c1 Name: c1 State: RUNNING PID: 2993 IP: 10.0.3.81 CPU use: 1.87 seconds BlkIO use: 18.83 MiB Memory use: 22.02 MiB KMem use: 0 bytes Link: vethLWC02H TX bytes: 1.04 KiB RX bytes: 1.12 KiB Total bytes: 2.16 KiBCopy the code

You can see the current system resource usage of the current C1 container.

With the lxc-attach command, you get a shell that interacts with the container, for example

   sudo lxc-attach -n c1
Copy the code

After attach succeeds, it enters a shell terminal,

	#root@c1:
Copy the code

Lxc-attach can be used to execute any command in the container, such as:

	sudo lxc-attach -n c1 ls
	sudo lxc-attach -n c1 top
Copy the code

You can package the APP as a Deb, copy the DEB to the appropriate location in container Rootfs, and finally install the DEB by executing the lxc-attach command.

Deb sudo lxc-attach -n c1 -- DPKG -i /home/ubuntu/app.deb install app.deb sudo lxc-attach -n c1 -- DPKG -r /home/ubuntu/app.deb / / uninstall the app. DebCopy the code

This terminal provides almost all of Ubuntu’s features, such as APT, but Ubuntu runs as a container under the Ubuntu Core physical host.

Finally, lxc-stop closes the container, and lxc-destroy deletes all configuration files associated with the container, including rootfs.

3.2 LXD

To be continued… .

3.3 BalenaOS

To be continued…

Embedded Linux container technology

Embedded Linux container technology

First, Linux container technology

Embedded Linux container

3. Experience container

3.1 the LXC

3.1.1 Hardware Environment

3.1.2 LXC installation

3.1.3 Creating a non-privileged Container

3.1.4 Creating a privilege container

3.2 LXD

3.3 BalenaOS

Related Posts

Summary of Java basic knowledge

Docker’s best practices on cloud platforms

Python3-opencv implements image processing — reads video files