This article is reprinted from the public account OceanBase
preface
At present, there are many articles about The functions, cases and stories of OceanBase. Friends who are interested in OceanBase would like to install a database to try. This article is to share how to set up an OceanBase cluster manually for beginners. This is also the first step in learning to understand how OceanBase clusters work. In the production environment, we have automatic operation and maintenance platform OCP, which is responsible for OceanBase cluster deployment and operation and maintenance in the production environment. At the same time, the download file provided on the official website also contains python scripts for automatic deployment of OceanBase cluster. After you are familiar with the installation principle, you can use automatic scripts to install OceanBase cluster.
OceanBase is a distributed database that exists in the form of a cluster and is a Share-nothing architecture. Each node is a normal x86 server that uses local domain (SSD), does not rely on shared storage, and does not have clustered file systems. All nodes can communicate with each other without direct connection. So OceanBase cluster setup is very simple compared to ORACLE RAC cluster.
Currently, OceanBase has a high requirement on memory resources. It is recommended that each node in the learning environment be larger than 32 GB, and 64 GB is better. In addition, the OceanBase cluster contains at least three nodes in an odd ascending number. It is better to have three machines, but if you have only one machine with enough memory (192G +), you can also start three OceanBase processes to simulate three nodes.
The OBServer to install
The OceanBase distributed database exists in the form of a cluster with at least three nodes. Each node is an Observer process. Multiple Observer processes on different nodes form a cluster to provide external services.
1. Plan the installation
The target of this installation is to build an OceanBase cluster with 2-2-2 architecture. In theory, this requires 6 machines, but in fact, I only have 3. So I would start two Observer processes on each machine to simulate two nodes.
The node planning is as follows. If you have only one machine with a lot of memory, you can have three or six OceanBase processes on it, taking care that the RPC Port and Connect Port do not duplicate or conflict with other application ports.
| Zone | IP | RPC Port | Connect Port |
|
Zone1 |
192.168.1.241 |
2882 |
2881 |
|
Zone1 |
192.168.1.241 |
3882 |
3881 |
|
Zone2 |
192.168.1.81 |
2882 |
2881 |
|
Zone2 |
192.168.1.81 |
3882 |
3881 |
|
Zone3 |
192.168.1.86 |
2882 |
2881 |
|
Zone3 |
192.168.1.86 |
3882 |
3881 |
Because OceanBase is a distributed database, application data may be distributed on any node. Therefore, applications need to use a reverse proxy OBProxy to access the OceanBase cluster. OBProxy can be installed on any machine, including OceanBase database nodes. Here I installed it on one of the nodes.
2. Prepare the machine environment
The OceanBase machine node environment is mainly used to install users, configure kernel parameters, firewall and SELinux Settings, time synchronization Settings, and user session limits.
-
Install the user
By default, OceanBase will be installed under user admin. You can choose to install it under another user (without using the RPM package).
It is generally recommended that the admin user be configured with sudo permission so that the root account is not required during the installation process.
Useradd adminpasswd admin# grant sudo permission chmod u+w /etc/sudoersvi /etc/sudoersroot ALL=(ALL) ALLadmin ALL=(ALL) ALLchmod u-w /etc/sudoers
Copy the code-
User Session Settings
You can run the ulimit command to view the default session limit. Modify the following files to make the default session limit changes permanent.
sudo vi /etc/security/limits.conf* soft nofile 655350* hard nofile 655350* soft stack 20480* hard stack 20480* soft nproc 655360* hard nproc 655360* soft core unlimited* hard core unlimited
Copy the codeExit the re-login and check that the Settings take effect
ulimit -a
Copy the code-
Kernel Parameter Configuration
Kernel parameters are mainly network and memory related Settings.
sudo vi /etc/sysctl.conffs.aio-max-nr = 65536net.core.somaxconn = 2048net.core.netdev_max_backlog = 10000net.core.rmem_default = 16777216net.core.wmem_default = 16777216net.core.rmem_max = 16777216net.core.wmem_max = 16777216net.ipv4.ip_local_port_range = 3500 65535net.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0net.ipv4.tcp_syncookies = 0net.ipv4.tcp_rmem = 4096 87380 16777216net.ipv4.tcp_wmem = 4096 65536 16777216net.ipv4.tcp_max_syn_backlog = 16384net.ipv4.tcp_fin_timeout = 15net.ipv4.tcp_max_syn_backlog = 16384net.ipv4.tcp_tw_reuse = 1net.ipv4.tcp_tw_recycle = 1net.ipv4.tcp_slow_start_after_idle=0vm.swappiness = 0kernel.core_pattern = /data/1/core-%e-%p-%tvm.min_free_kbytes = 2097152vm.max_map_count=655360
Copy the codeRun the following command to check whether the modification takes effect
sudo sysctl -p
Copy the code-
Disable the firewall and SELinux
The firewall and SELinux security features of Linux may cause abnormal communication between OceanBase nodes, so you are advised to disable them
sudo systemctl disable firewalldsudo systemctl stop firewalldsudo systemctl status firewalldsudo vi /etc/selinux/configSELINUX=disabledsudo setenforce 0
Copy the code-
Time synchronization on a node
OceanBase is a distributed database. The time of each node must be synchronized with the time error within 50ms. If the node time error is too large, the OceanBase cluster will fail to be initialized later or the existing OceanBase cluster will become abnormal.
Time synchronization between nodes is ensured by Linux NTP synchronization. To check the time error, run the clockdiff command
Clockdiff 192.168.1.86 clockdiff 192.168.1.81 clockdiff 192.168.1.241
Copy the codeConfigure NTP synchronization for nodes
Sudo vi/etc/NTP. Confserver 192.168.1.239
Copy the codeStop the NTPD service, forcibly synchronize time, and then start the NTPD service. Subsequent time synchronization relies on the NTPD service.
sudo service ntpd stop; Sudo ntpdate 192.168.1.239; sudo service ntpd start; ntpq -4p
Copy the code
3. Software installation
-
Software catalog
At present, oceanbase.alipay.com provides OCP trial version download, which contains many OCP and OB files, manual installation only need two RPM files: OBProxy and Oceanbase. The acquisition method is as follows:
wget https://gw.alipayobjects.com/os/downloads/ossupload/ocp-release.zipunzip ocp-release.ziptar zxvf ocp-setup.tar.gzcd Ocp_yhls - LRTH obproxy - 1.3.3-1506155. El7. X86_64. RPM oceanbase - 1.4.60-1571952. El7. X86_64. RPM
Copy the codeLet’s take a look at the contents of oceanbase’s RPM package.
Rpm2cpio oceanbase 1.4.60-1571952. El7. X86_64. RPM | cpio - divcd home/admin; The tar CZVF ob1.4.6. Tar. Gz oceanbase/CD ~ / SCP home/admin/ob1.4.6 tar. Gz 192.168.1.241: / TMP /; SCP home/admin/ob1.4.6. Tar. Gz 192.168.1.81: / TMP /; SCP home/admin/ob1.4.6. Tar. Gz 192.168.1.86: / TMP /;
Copy the codeYou can see the directory structure as follows. The bin directory contains an executable file observer, and the etc directory is used to store configuration and related scripts.
If you install the RPM package directly, it will be installed in the root directory of the admin user. Since we are starting two Observer processes on the same machine, the software directories are kept separate. Instead, unzip the RPM package and copy it to two directories. The owner of the directory is admin.
su - adminmkdir -p node1 node2cd ~/node1; Tar ZXVF/TMP /ob1.4.6.tar.gz -c. CD ~/node2; tar ZXVF/TMP /ob1.4.6.tar.gz -c. Tar ZXVF/TMP /ob1.4.6.tar.gz -c. CD ~/; tree
Copy the code-
Storage directory
The OceanBase storage directory is in the Store folder under the software directory. The storage directory contains directories for data files and related log files. For performance reasons, separate file systems are required for both the data file directory and the associated log file directory (it is better if the underlying disk is a separate disk). Then map the actual directory to the Store directory in the software directory.
su - adminmkdir -p /data/data/1/obdemo/cd /data/data/1/obdemo/mkdir -p etc3 sort_dir sstablemkdir -p /data/log/log1/obdemo/cd /data/log/log1/obdemo/mkdir -p clog etc2 ilog oob_clog slogmkdir -p /home/admin/node1/oceanbase/store/obdemocd /home/admin/node1/oceanbase/store/obdemo/ln -s /data/data/1/obdemo/sort_dir /home/admin/node1/oceanbase/store/obdemo/sort_dirln -s /data/data/1/obdemo/sstable /home/admin/node1/oceanbase/store/obdemo/sstableln -s /data/log/log1/obdemo/clog /home/admin/node1/oceanbase/store/obdemo/clogln -s /data/log/log1/obdemo/ilog /home/admin/node1/oceanbase/store/obdemo/ilogln -s /data/log/log1/obdemo/oob_clog /home/admin/node1/oceanbase/store/obdemo/oob_clogln -s /data/log/log1/obdemo/slog /home/admin/node1/oceanbase/store/obdemo/slogtree ~/node1 -uhmkdir -p /data/data/2/obdemo/cd /data/data/2/obdemo/mkdir -p etc3 sort_dir sstablemkdir -p /data/log/log2/obdemo/cd /data/log/log2/obdemo/mkdir -p clog etc2 ilog oob_clog slogmkdir -p /home/admin/node2/oceanbase/store/obdemocd /home/admin/node2/oceanbase/store/obdemo/ln -s /data/data/2/obdemo/sort_dir /home/admin/node2/oceanbase/store/obdemo/sort_dirln -s /data/data/2/obdemo/sstable /home/admin/node2/oceanbase/store/obdemo/sstableln -s /data/log/log2/obdemo/clog /home/admin/node2/oceanbase/store/obdemo/clogln -s /data/log/log2/obdemo/ilog /home/admin/node2/oceanbase/store/obdemo/ilogln -s /data/log/log2/obdemo/oob_clog /home/admin/node2/oceanbase/store/obdemo/oob_clogln -s /data/log/log2/obdemo/slog /home/admin/node2/oceanbase/store/obdemo/slogtree /home/admin/node2/oceanbase/store/obdemo/ -uhtree ~/node2 -uh
Copy the codeThe associated catalog for this complete OBServer process is as follows:
-
Rely on
RPMPackage installation
OceanBase compresses data, so it relies on RPM files with two compression protocols: snappy. X86_64 and lzo.x86_64.
rpm -q snappy.x86_64 lzo.x86_64yum -y install snappy.x86_64 lzo.x86_64
Copy the code
The OceanBase cluster is initialized
1. The OBServer
-
Launch parameters
Start the first Observer process on each node, listening on ports 2881 and 2882. The gray italics in the figure below are adjusted according to actual conditions. Change the zone name, IP address, and network adapter name as required.
192.168.1.241:2881:2882 zone1 Change the zone, IP address, and nic name
cd /home/admin/node1/oceanbase && /home/admin/node1/oceanbase/bin/observer -i eth0 -P 2882 -p 2881 -z zone1 -d / home/admin/node1 oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=100G,datafile_size=200G,config_additional_dir=/data/data/1/obdemo/etc3; /data/log/log1/obdemo/etc2" ps -ef| grep observer vi log/observer.log
Copy the codeThe boot names of the other two nodes are as follows:
192.168.1.81:2881:2882 zone2 Change the zone, IP address, and nic name
cd /home/admin/node1/oceanbase && /home/admin/node1/oceanbase/bin/observer -i eth3 -P 2882 -p 2881 -z zone2 -d / home/admin/node1 oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=100G,datafile_size=200G,config_additional_dir=/data/data/1/obdemo/etc3; /data/log/log1/obdemo/etc2"ps -ef|grep observervi log/observer.log
Copy the code192.168.1.86:2881:2882 zone3 Change the zone, IP address, and nic name
cd /home/admin/node1/oceanbase && /home/admin/node1/oceanbase/bin/observer -i eth3 -P 2882 -p 2881 -z zone3 -d / home/admin/node1 oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=100G,datafile_size=200G,config_additional_dir=/data/data/1/obdemo/etc3; /data/log/log1/obdemo/etc2"ps -ef|grep observervi log/observer.log
Copy the code2. Initialize the cluster
After all three OBServer nodes have been started, this step of cluster initialization command is critical and it is a matter of success or failure.
Log in to any node using the mysql command, and then run the bootstrap command. Note There are no Spaces or whitespace in the command.
Mysql -h192.168.1.241-uroot -p2881 -palter system bootstrap ZONE 'zone1' SERVER '192.168.1.241:2882', ZONE 'zone2' SERVER '192.168.1.81:2882', ZONE 'zone3' SERVER '192.168.1.86:2882';
Copy the codeAbout 10 seconds later, the cluster starts successfully. Exit and log in to the cluster again. The username format of the login command needs to be changed. Change the default sys tenant administrator root@sys.
Mysql -h192.168.1.241 -uroot@sys -p2881 -p -c -a oceanbasealter user root identified by 'rootPWd ';
Copy the codeExit Password authentication and view cluster resource information.
Mysql -h192.168.1.241 -u root@sys -p2881 -prootpwd -c -a Oceanbase select zone, sVR_IP, inner_port, CPU_total, cpu_assigned, cpu_assigned_percent cpu_ass_percent, round(mem_total/1024/1024/1024 ) mem_total_gb, round(mem_assigned/1024/1024/1024 ) mem_ass_gb, round(disk_total/1024/1024/1024 ) disk_total_gb, unit_num, Substr (build_version,1,6) versionfrom __all_virtual_server_stat order by zone, svr_ip, inner_port;
Copy the code3. Expand the cluster
So far, it’s still builtThe 1-1-1Add three more nodes to the OceanBase clusterThe 2-2-2Layout. I’m just going to start the second one on three machinesobserverProcess, listening on ports 3881 and 3882.
-
The node to
192.168.1.241:3881:3882 zone1 Change the node zone, IP address, port, and nic name
cd /home/admin/node2/oceanbase && /home/admin/node2/oceanbase/bin/observer -i eth0 -P 3882 -p 3881 -z zone1 -d / home/admin / 2 / oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=61440M,datafile_size=100G,config_additional_dir=/data/data/2/obdemo/etc3; /data/log/log2/obdemo/etc2"ps -ef|grep observervi log/observer.log
Copy the code192.168.1.81:2881:2882 zone2 Change the node zone, IP address, port, and nic name
cd /home/admin/node2/oceanbase && /home/admin/node2/oceanbase/bin/observer -i eth3 -P 3882 -p 3881 -z zone2 -d / home/admin / 2 / oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=61440M,datafile_size=100G,config_additional_dir=/data/data/2/obdemo/etc3; /data/log/log2/obdemo/etc2"ps -ef|grep observervi log/observer.log
Copy the code192.168.1.86:2881:2882 zone3 Change the node zone, IP address, port, and nic name
cd /home/admin/node2/oceanbase && /home/admin/node2/oceanbase/bin/observer -i eth3 -P 3882 -p 3881 -z zone3 -d / home/admin / 2 / oceanbase/store/obdemo -r '192.168.1.241:2882-2881; 192.168.1.81:2882-2881; 192.168.1.86:2882:2881' -c 20190423 -n obdemo -o "cpu_count=24,memory_limit=61440M,datafile_size=100G,config_additional_dir=/data/data/2/obdemo/etc3; /data/log/log2/obdemo/etc2"ps -ef|grep observervi log/observer.log
Copy the code-
The cluster expansion
Add the new three nodes to the OceanBase cluster. Ensure that the zone name and IP address are correct.
Alter system add server '192.168.1.241:3882' zone 'zone1'; Alter system add server '192.168.1.81:3882' zone 'zone2'; Alter system add server '192.168.1.86:3882' zone 'zone3'; select zone, svr_ip, inner_port, cpu_total, cpu_assigned, cpu_assigned_percent cpu_ass_percent, round(mem_total/1024/1024/1024) mem_total_gb, round(mem_assigned/1024/1024/1024) mem_ass_gb, round(disk_total/1024/1024/1024) disk_total_gb, unit_num, Substr (build_version,1,6) versionfrom __all_virtual_server_statorder by zone, svr_ip, inner_port;
Copy the code4. Install and start OBProxy
OceanBase is a distributed database. The data of an application may be distributed on any node and not fixed. Therefore, it is impossible for an application to know which node the data is on or record all node IP addresses. A reverse proxy, OBProxy, is required to provide SQL routing functionality in front of the OceanBase cluster. Applications access the OceanBase cluster through OBProxy, which is the database representative.
-
OBProxy software installation
Only one OBproxy needs to be installed on a node, so you can install it directly using the RPM package.
su - adminmkdir -p /home/admin/logs/obproxy/log /home/admin/logs/obproxy/minidumpsudo rpm -ivh Obproxy - 1.3.3-1506155. El7. X86_64. RPM
Copy the code-
OBProxy dedicated user
OBProxy needs to communicate with the OceanBase cluster, so you need to create an account in advance with the SYS tenant.
CREATE USER proxyro IDENTIFIED BY password '*e9c2bcdc178a99b7b08dd25db58ded2ee5bff050' ; GRANT SELECT ON *.* to proxyro; show grants for proxyro;
Copy the code-
OBProxy start
OBProxy startup is similar to OBServer startup
Note that -r specifies the rootService list address in a slightly different format and does not require RPC Port information. -p Specifies the listening port of OBProxy.
CD/opt/taobao/install/obproxy && bin/obproxy - r "192.168.1.241:2881; 192.168.1.81:2881; 192.168.1.86:2881-2883 - p o "" enable_strict_kernel_release = false, enable_cluster_checkout = false" - c obdemops - ef | grep obproxy
Copy the codeThere are two formats for the command to connect to the OceanBase cluster through OBProxy. The difference lies in the format of the user name. For example, [User name]@[Tenant name]#[Cluster name] or [cluster name]:[Tenant name]:[User name].
Mysql -h192.168.1.241 -uroot@sys# obdemo-p2883 -prootpwd -c -c A oceanbase or mysql -h192.168.1.241 -uobdemo:sys:root -p2883 -prootpwd -c -A oceanbase
Copy the code
OceanBase Cluster o&M
One significant difference between OceanBase distributed database and traditional database or other distributed database products is the idea of resource management. The OceanBase cluster aggregates the resources (CPU, memory, and space) of multiple hosts into a large pool and allocates resources of specified specifications to a specific service. This is provided for business use by the tenant, also known as the instance.
The following shows how to create a tenant.
1. Create a tenant
-
Defining resource Specifications
Mysql -h192.168.1.241 -uroot@sys#obdemo -p2883 -prootpwd -c -a oceanbasecreate resource unit unit_4c20g512g, max_CPU =4, max_memory='20G', min_memory='10G', max_iops=10000, min_iops=1000, max_session_num=1000000, max_disk_size=53687091200; create resource unit unit_8c40g1024g, max_cpu=8, max_memory='40G', min_memory='20G', max_iops=20000, min_iops=5000, max_session_num=1000000, max_disk_size=107374182400; create resource unit unit_16c80g2048g, max_cpu=16, max_memory='80G', min_memory='40G', max_iops=50000, min_iops=10000, max_session_num=1000000, max_disk_size=214748364800; select unit_config_id,name,max_cpu,min_cpu,round(max_memory/1024/1024/1024) max_mem_gb, round(min_memory/1024/1024/1024) min_mem_gb, round(max_disk_size/1024/1024/1024) max_disk_size_gbfrom __all_unit_configorder by unit_config_id;
Copy the code-
Creating a Resource Pool
Mysql -h192.168.1.241 -uroot@sys# obdemo-p2883 -prootpwd -c -a OceanBasecreate resource pool pool_demo unit = 'unit_4c20g512g', unit_num = 1; select resource_pool_id, name,unit_count, unit_config_id, zone_list, tenant_id, gmt_modifiedfrom __all_resource_pool order by resource_pool_id ;
Copy the code-
Create a tenant
Mysql -h192.168.1.241 -uroot@sys# obdemo-p2883 -prootpwd -c -a Oceanbasecreate tenant t_obdemo resource_pool_list=('pool_demo'); select tenant_id, tenant_name, zone_list, locality ,gmt_modified from __all_tenant;
Copy the code-
Log on to the tenant
Tenant Connection The sys tenant is disconnected and the user tenant is connected
Mysql -h192.168.1.241 -uroot@t_obdemo#obdemo -p2883 Databases; create database sysbenchtest; grant all privileges on sysbenchtest.* to testuser@'%' identified by 'testpwd';
Copy the code2. Expand the capacity of the tenant
-
Check the resource capacity of the tenant
Mysql -h192.168.1.241 -uroot@t_obdemo# obdemo-p2883 oceanbase a-pselect tenant_id,tenant_name, unit_id, zone, svr_ip,svr_port,max_cpu,min_cpu,round(max_memory/1024/1024/1024) max_mem_gb, round(min_memory/1024/1024/1024) min_mem_gb, round(max_disk_size/1024/1024/1024) max_disk_size_gbfrom gv$unit;
Copy the codeIf the service finds that the database is slow and the bottleneck is resources, expand the resource capacity of the tenant.
-
The tenant expansion
There are two ways to expand the capacity of a tenant: one is to improve the specifications of resource units, the other is to increase the number of resource units.
Mysql -h192.168.1.241 -u obdemo:sys:root -p2883 -prootpwd -c -a Oceanbase ALTER resource Pool pool_demo unit = 'unit_8c40g1024g' ; alter resource pool pool_demo unit_num = 2;
Copy the codeShrinkage capacity in the same way
alter resource pool pool_demo unit_num = 1;
Copy the code3. Set cluster parameters
At this point, the OceanBase cluster and tenants are set up. In order to prevent OceanBase from becoming unstable due to resource shortage, some parameters need to be configured, similar to parameter changes in ORACLE.
-
Modifying Cluster Parameters
Set log parameters
Mysql -h192.168.1.241 -u obdemo:sys:root -p2883 -prootpwd -c -a Oceanbase -- observer log Self-clearing alter system set enable_syslog_recycle= True; alter system set max_syslog_file_count= 5; show parameters where name in ( 'enable_syslog_recycle', 'max_syslog_file_count' );
Copy the codeConfigure OBProxy parameters
Alter PROxyConfig set enable_metadb_used=False; alter proxyconfig set enable_proxy_scramble=True; alter proxyconfig set proxy_mem_limited=2G; alter proxyconfig set log_dir_size_threshold=10G;
Copy the codeSet freeze merge parameters
Alter SYSTEM set ENABLE_MERGE_BY_TURN = false; alter system set minor_freeze_times=3; alter system set minor_warm_up_duration_time='300s';
Copy the code-
Tenant variable modification
OceanBase also supports configuration of tenant behavior through changes to tenant variables. Similar to variable modification in MySQL.
Mysql -h192.168.1.241 -u root@t_obdemo#obdemo -p2883 oceanBase -a -pset global ob_query_timeout=100000000;
Copy the code
Clearing the OceanBase cluster
If an error occurred during cluster initialization, you need to clean up all the steps and start again.
# # # OceanBase installation environment clean ` ` ` bashpkill observerSleep 10 RPM - qa | grep oceanbaserpm - qa | grep obproxysudo RPM - esu - admin/bin/rm -rf ~/node1 ~/node2/bin/rm -rf /data/data/* /data/log/*
Copy the code
OceanBase learning materials
So much for the Manual setup of the OceanBase cluster. Is it much simpler than ORACLE RAC? Due to limited time and space, the OceanBase principle involved in the installation is not explained in detail. For those interested, check out OceanBase’s other learning resources. In the future, we will continue to share OceanBase operation and maintenance experience live. Please stay tuned.
-
OceanBase website: https://tech.antfin.com/products/OCEANBASE
-
Wechat official account: OceanBase
-
OceanBase nailing AC group: Scan codes into the group
-
OceanBase Cloud Community Forum:
https://bbs.aliyun.com/thread/439.html
Please scan the qr code below and reply to “0423” to obtain the live PPT
Click “Read the original” to see the video review!
