This is the 18th day of my participation in Gwen Challenge. For more information, see: Gwen Challenge 😄


  \color{red}{~}

Docker required installation software – Basic version

This is a basic version, we can install mysql, mount the disk, do not configure environment variables, Docker container management visual interface is very easy to use, build their own private warehouse these are our necessary knowledge ！！！！！

Dokcer install rancher

Docker run -d -p 80:80 -p 443:443 Rancher /rancher:v2.4.0 // rancher install K8S add cluster ->custom-> copy command on at least 4G machine - Install successful --25分 --37分Copy the code

Dokcer install Redis

Docker pull redis:latest docker start redis docker run -p 6379:6379 --name redis -v /opt/docker_redis/redis.conf:/etc/redis/redis.conf -v /opt/docker_redis/data:/data -d redis redis-server /etc/redis/redis.conf --appendonly yes --requirepass 123456 Command Description -p 6379:6379 Port mapping: the first port indicates the host port and the last port indicates the container port. --name redis Specifies the name of the container, which is easy to view and operate. -v Indicates the host part and: indicates the container part. -d redis indicates to start the redis in the background redis redis-server /etc/redis/redis. Conf Start the redis in the configuration file and load the conf file in the container. Eventually found is mounted directory/usr/local/docker/redis conf - appendonly yes open redis persistence - requirepass 123456 set the password for 123456 to enter: Docker exec it redis /bin/bash Access: docker exec it redis /bin/bash redis-cli -p 6379 Obtain the password: config get requirepass Enter the password: Auth 123456 Cache penetration, not found (set null cache, Bloom filter) and avalanche (multi-level cache) cache breakdown, found more, expiration: expiration time scattered Settings, hot data does not expireCopy the code

Dokcer install influxDB

Json {"exec-opts": [" native-cgroupdriver =systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100 m"}, "insecure - registries:" [] "192.168.180.105:1180", "registry-mirrors":["https://6kx4zyno.mirror.aliyuncs.com"] } docker run -d -p 8083:8083 -p 8086:8086 --name my_influxdb Influxdb docker exec it my_influxdb bash CD /usr/bin. /influx Enter influxDB client show databases create database jmeter; Use jmeter select * from jmeter add: When we insert a piece of data when measurement doesn't exist, Measurement INSERT add_test,name=YiHui,phone=110 user_id=20,email="[email protected]" Measurement is add_test, tag is name,phone, Field is user_id,email show measurements query table SELECT * from add_test time email name phone user_id ---- ----- ---- ----- ------- 1604641039680640423 [email protected] YiHui 110 2 show tag keys from add_test; name: Add_test tagKey ------ name phone Insert + measurement + "," + tag=value,tag=value + + field=value,field=value Tags are strings and do not need quotation marks to wrap values (often queried, group by). If fields are strings, quotation marks are required (used for function calculation, Non-string) type summary: SELECT * FROM "TB_TRACE" WHERE vno = 'L71692' and time >= '2021-01-22 07:46:05' and time <= '2021-01-22 08:46:05 Method Example Float number user_id=21 int number I age= 18I Boolean True /false boy=true String "" or "email=" [email protected] "1. The timestamp specifies that when writing data without specifying a time, the current time will be automatically used to complete the data. If you need to specify a time by yourself, you can add the timestamp at the end. Ns insert add_test,name=YiHui,phone=110 user_id=22,email="[email protected]",age=18i,boy=true 1564150279123000000 2 Specify a save policy If no save policy is specified before data insertion, the data is written to the default save policy. We know that a database can have multiple save policies, and a single measurement can hold data of different save policies. If you need to specify save policies when writing data, you can use insert into save policies. Show Retention policies on test Check the existing policies of the database name Duration shardGroupDuration replicaN default ---- -------- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- autogen 0 s 168 h0m0s 1 true new strategy to CREATE the RETENTION POLICY "1 _d" ON "jmeter" DURATION 1d REPLICATION 1 DEFAULT This example adds a 2-hour policy to the JMeter library named 2_hours, duration is 2 hours, duplicate is 1, and set to the DEFAULT policy. ALTER RETENTION POLICY "2_hours" ON "telegraf" DURATION 4h DEFAULT ALTER RETENTION POLICY "TRACE_RETENTION_POLICY" ON "SMART_DATA" DURATION 8760h DEFAULT Retention POLICY: DROP Retention POLICY "2_hours" ON "telegraf" Retention POLICY for inserting data tapes: Insert into "1_d" add_test,name=YiHui2,phone=911 user_id=23,email="[email protected]",age=18i,boy=true 1604642513123000000 Query the current policy data: select * from "1_d". Add_test Backup data: Influxd backup -portable/TMP /data/total CD/TMP /data/total Restore to a new database yhHBlog_bk. Run the following command to recover to an existing DB: influxd restorest-portable -db yhhblog -newdb yhHBlog_bk yhHBlog_per https://blog.hhui.top/hexblog/2018/07/27/180727-%E6%97%B6%E5%BA%8F%E6%95%B0%E6%8D%AE%E5%BA%93InfluxDB%E4%B9%8B%E5%A4%87% E4%BB%BD%E5%92%8C%E6%81%A2%E5%A4%8D%E7%AD%96%E7%95%A5/ One of the strategies used is to backup to a temporary DB first; 1. Since the tags in the influxDB are all string types, indexes are established. Therefore, the efficiency of tag-based query is generally better than that of field query. = 2. There is no in query in the influxdb, and different query conditions can be connected using AND /or, indicating that batch insertion can be achieved by Java code if either or is met simultaneously: Visualization tool influxDBstudio: directly in the word seach 11, dokcer installation ES, kibana docker pull docker. IO/elasticsearch: 7.1.1 docker run - d - e "ES_JAVA_OPTS=-Xms512m -Xmx512m" --name es2 -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" 5acf0e8da90b docker logs 676164fb24b3 docker exec -it es2 /bin/bash elasticsearch.yml cluster.name: "qfcwx-cluster" network.host: 0.0.0.0 http.cers. enabled: true http.cers. allow-origin: "*" cluster.name: user-defined cluster name. Network. host: IP address bound to the current ES node. The default value is 127.0.0.1. Http.cers. enabled: Indicates whether cross-domain is supported. The default value is false. Http.coron. allow-origin: When cross-domain is enabled, the default value is *, which indicates that all domain names are supported. If only certain websites are allowed to access, the regular expression can be used. Docker run - name kibana ELASTICSEARCH_URL = http://47.104.130.105:9200 - p - e 5601:5601 - d kibana: 7.1.1 into the container to modify kibana. Yml Example Modify the IP port connected to esCopy the code

Dokcer install kafka panel

vim docker-compose.yml version: "3" services: kafdrop: image: obsidiandynamics/kafdrop restart: "no" ports: - "8080:9000" environment: KAFKA_BROKERCONNECT: Docker-compose up -d for Linux: compose kafka CD /opt/supp_app/kafka_2.13-2.6.0/bin./kafka-console-producer.sh - the bootstrap -server 172.17.47.173: $9092172.17. 47.174:9092172.17. 47.175:9092 - topic queue_topic . / kafka - the console - consumer. Sh -- -- the bootstrap - server 172.17.47.173: $9092172.17. 47.174:9092172.17. 47.175:9092 - topic Queue_topic {"vehicleNo":" ruq327EB ","fenceId":"7980862197161687045","entryTime":"2021-01-19 08:13:33","leaveTime":""}Copy the code

Docker install hbase

Docker pull Harisekhon /hbase:1.3 docker run -d --name hbase001 -P 16010:16010 Harisekhon /hbase:1.3 Docker execit hbase001 bashCopy the code

Docker install Kylin

Kylin.apache.org/cn/docs/ins…

Docker pull Apachekylin/Apache-kylin-standalone :3.1.0 Docker run -d \ -m 8G \ -p 7070:7070 \ -p 8088:8088 \ -p 50070:50070 \ -p 8032:8032 \ -p 8042:8042 \ -p 16010:16010 \ Apachekylin/Apache-kylin-standalone :3.1.0 and run automatically $KYLIN_HOME/bin/sample.sh and create "kylin_streaming_ topic" topic in Kafka and keep sending data to that topic. This is so that users can start the container and experience building cubes and querying them in batch and stream fashion. After the container is started, you can run the "docker exec -it <container_id> bash" command to access the container. Of course, because we already have designated container port mapping to the local port, we can directly in the machine in the browser Open the page of each service, such as: Kylin page: http://127.0.0.1:7070/kylin/login HDFS NameNode page: http://127.0.0.1:50070 YARN ResourceManager: http://127.0.0.1:8088 HBase: http://127.0.0.1:16010Copy the code

Use docker to operate basic commands of Redis, influxDB detailed operation, hbase and Klin are all necessary and commonly used software, Rancher is a very useful software, ES and Kibana are all necessary to master.

Thank you for reading this, if this article is well written and if you feel there is something to it

Ask for a thumbs up 👍 ask for attention ❤️ ask for share 👥 for 8 abs I really very useful!!

If there are any mistakes in this blog, please comment, thank you very much! ❤ ️ ❤ ️ ❤ ️ ❤ ️