Linux set up virtual memory teaching and actual combat

introduce

In our own server environment, we usually buy 1g of memory, but when there are too many things in the server, it will lead to insufficient memory. This paper will simulate a real memory insufficient situation, how to modify the virtual memory to make the system run normally. Our environment is set up for ElasticSearch, but we only have 1 GB of memory on the server. Here is how to change 1 GB of virtual memory to 4 GB.

Build the ElasticSearch environment

Now our server environment is empty, there is nothing, we will upload ElasticSearch to the server, then install the JDK and ElasticSearch.

Install the JDK

The setup tutorial will be updated later (this article focuses on setting up virtual memory, which is installed to simulate an out-of-memory state)

Install the ElasticSearch

The install link will be updated later (this article focuses on setting up virtual memory, which is installed to simulate an out-of-memory state)

Start the ElasticSearch

1. ElasticSearch: error: the server does not have enough memory for ElasticSearch.

[esyonghu@localhost elasticSearch -6.4.0]$./bin/ elasticSearch [1] 3228 [esyonghu@localhost elasticSearch -6.4.0]$Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x000000008a660000, 1973026816, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1973026816 bytes for committing reserved memory.
# An error report file with more information is saved as:
# logs/hs_err_pid3228.log[esyonghu @ localhost elasticsearch - 6.4.0] $Copy the code

2. View the memory of our server using the commandfreeWe can see that the memory of our server is 1 GB, so we need to modify the virtual memory to solve the problem.

[esyonghu@localhost elasticSearch-6.4.0]$free -m total Used free shared buffers cached Mem: [esyonghu@localhost elasticSearch-6.4.0]$Copy the code

Creating a swap file

1. Enter the/usrdirectory

[root@localhost usr]$ pwd
/usr
[root@localhost usr]$ 
Copy the code

2. createswapFolder and enter the folder

[root@localhost usr]# mkdir swap
[root@localhost usr]# cd swap/
[root@localhost swap]# pwd
/usr/swap
[root@localhost swap]# 
Copy the code

3. createswapfileFile, use the commanddd if=/dev/zero of=/usr/swap/swapfile bs=1M count=4096

[root@localhost swap]# dd if=/dev/zero of=/usr/swap/swapfile bs=1M count=40964096+0 read record 4096+0 write record 4294967296 bytes (4.3 GB) copied 15.7479 seconds 273 MB/ SEC [root@localhost swap]#
Copy the code

Viewing swap files

1. Using the commanddu -sh /usr/swap/swapfile, you can see that the swap file we created is 4G

[root@localhost swap]# du -sh /usr/swap/swapfile4.1 G/usr/swap/swapfile [root @ localhost swap]# 
Copy the code

Set the target to the swap partition file

1. Using the commandmkswap /usr/swap/swapfileSet swapfile to swap partition file

[root@localhost swap]# mkswap /usr/swap/swapfile
mkswap: /usr/swap/swapfile: warning: don't erase bootbits sectors on whole disk. Use -f to force. Setting up swapspace version 1, size = 4194300 KiB no label, UUID=5bd241ff-5375-449d-9975-5fdd429df784 [root@localhost swap]#Copy the code

Activate swap and immediately enable swap files

1. Using the commandswapon /usr/swap/swapfile

[root@localhost swap]# swapon /usr/swap/swapfile
[root@localhost swap]#
Copy the code

2. Using the commandfree -mTo view the current memory, you can see that the Swap partition has become 4095MB, that is, 4G memory.

[root@localhost swap]# free -m
             total       used       free     shared    buffers     cached
Mem:           980        910         70          3          8        575
-/+ buffers/cache:        326        654
Swap:         4095          0       4095
[root@localhost swap]#
Copy the code

Set automatic startup virtual memory, inetc/fstabAdd the following command to the file

1. Open the /etc/fstab file using the Vim editor

2. Add the following content to the file

/usr/swap/swapfile swap swap defaults 0 0
Copy the code

Run the reboot command to restart the server

1. The inputrebootCommand to restart

	[root@localhost swap]# reboot

	Broadcast message from [email protected]
		(/dev/pts/1) at 3:56 ...

	The system is going down for reboot NOW!
	[root@localhost swap]Connection to 192.168.136.142 closed by remote hostConnection to 192.168.136.142 closed. [Process completed]Copy the code

2. After the restart, use the free -m command to see if the memory is now hanging on.

[root@localhost swap]# free -m
             total       used       free     shared    buffers     cached
Mem:           980        910         70          3          8        575
-/+ buffers/cache:        326        654
Swap:         4095          0       4095
Copy the code

Start ElasticSearch again to see if the out-of-memory error is reported again

1. Elasticsearch (esyonghu, esyonghu, esyonghu, Esyonghu, Esyonghu, Esyonghu, Esyonghu)

[esyonghu@localhost elasticSearch -6.4.0]$./bin/ elasticSearch & [1] 2898 [esyonghu@localhost elasticSearch -6.4.0]$ [2019-03-06T04.00:24.841][INFO][O.E.n.ode][] initializing... [INFO][O.e.e.nodeenvironment][dMy5nR5] using [1] Data paths, mounts [[/ (rootfs)]], mounts [/ rootfs] Net usable_space [7.6gb], net total_space [17.3gb], Types [rootfs] [2019-03-06T04:00:24,928][INFO][O.e.E.nodeEnvironment][dMy5nR5] Heap size [1.9 GB], compressed ordinary object pointers [true[2019-03-06T04:00:25,018][INFO][O.e.n.node][dMy5nR5] Node Name Derived from node ID [dmy5nr5fthabB-q2t0txda];set[node.name] Override [2019-03-06T04:00:25,018][INFO][O.E.n.ode][dMy5nR5] version[6.4.0], PID [2898], Build/default/tar / 595516/2018-08-17 e T23:18:47. 308994 z], OS/Linux / 2.6.32-696. The el6. X86_64 / amd64]. JVM[Oracle Corporation/Java HotSpot(TM) 64-bit Server VM/1.8.0_181/25.181-b13] [2019-03-06T04:00:25,018][INFO ][o.e.n.Node ] [dMy5nR5] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.24Q3S9AE, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, - XX: GCLogFileSize = 64 m, - Des. Path. The home = / home/esyonghu/elasticsearch - 6.4.0, - Des. Path. The conf = / home/esyonghu/elasticsearch - 6.4.0 / config, - Des. Distribution. The flavor = default, -des.distribution. type=tar] [2019-03-06T04:00:28.022][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [2019-03-06T04:00:28,023][INFO][o.E.p.puginsService][dMy5nR5] Loaded Module [analysis-common] [2019-03-06T04:00:28,023][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [ingest-common] [2019-03-06T04:00:28.023][INFO][o.e.P.luginsService][dMy5nR5] Loaded Module [lang-expression] [2019-03-06T04:00:28.023][INFO][O.E.P.LuginsService][dMy5nR5] Loaded Module [Mustache] [2019-03-06T04:00:28.023][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [lang-painless] [2019-03-06T04:00:28.023][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [2019-03-06T04:00:28.023][INFO][o.e.p.pluginsService][dMy5nR5] Loaded Module [parent-join] [2019-03-06T04:00:28.023][INFO][o.e.p.pluginsService][dMy5nR5] Loaded Module [percolator] [2019-03-06T04:00:28.023][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [rank-eval] [2019-03-06T04:00:28.023][INFO][o.e.p.pluginsService][dMy5nR5] Loaded Module [INFO][2019-03-06T04:00:28.023 [o.e.p.luginsService][dMy5nR5] Loaded Module [repository-url] [2019-03-06t04:00:28.023][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [repository-url] [o.e.p.luginsService [o.e.p.pluginsService][dMy5nR5] loaded module as it was loaded [tribe] [2019-03-06T04:00:28,024][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [x-pack-core] [2019-03-06T04:00:28.024][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Module [X-pack -deprecation] [2019-03-06T04:00:28,024][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Module [x-pack-graph] [2019-03-06T04:00:28.024][INFO][o.e.P.luginsService][dMy5nR5] Loaded Module [x-pack-logstash] [2019-03-06T04:00:28,024][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Module [x-pack-ml] [2019-03-06T04:00:28.024][INFO][o.e.p.pluginsService][dMy5nR5] Loaded Module [x-pack-monitoring] [2019-03-06T04:00:28,024][INFO][o.e.p.pluginsService][dMy5nR5] Loaded Module [x-pack-rollup] [2019-03-06T04:00:28.024][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Module [x-pack-Security] [2019-03-06T04:00:28.024][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [x-pack-sql] [2019-03-06T04:00:28,024][INFO][o.e.p.luginsService][dMy5nR5] Loaded Module [x-pack-upgrade] [2019-03-06T04:00:28,024][INFO][o.e.p.pluginsService] Loaded Module [x-pack-watcher] [2019-03-06T04:00:28,025][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Plugin [analysis-ik] [2019-03-06T04:00:28,025][INFO][o.e.p.pulginsService][dMy5nR5] Loaded Plugin [the 2019-03-06 T04:00:31, 315] [INFO] [O.E.X.S.A.S.F ileRolesStore] [dMy5nR5] parsed [0] roles from the file [/ home/esyonghu/elasticsearch - 6.4.0 / config/roles. Yml] [the 2019-03-06 T04:00:32, 017] [INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/2947] [Main.cc@109] controller (64 bit): Version 6.4.0 (Build CF8246175EFFF5) Copyright (C) 2018 Elasticsearch BV [2019-03-06T04:00:32.495][DEBUG][O.E.A. ctionModule] Using REST Wrapper from plugin Org. Elasticsearch. Xpack. Security. Security [the 2019-03-06 T04:00:32, 768] [INFO] [O.E.D.D iscoveryModule] [dMy5nR5] using discoverytype[zen] [2019-03-06T04:00:33,628][INFO][o.E.n.mode][dMy5nR5] Initialized [2019-03-06T04:00:33628][INFO][O.E.n.mode] [dMy5nR5] starting ... [2019-03-06T04:00:33.860][INFO][O.E.T. ransportService][dMy5nR5] publish_address {192.168.136.142:9300}, Bound_addresses {[::]:9300} [2019-03-06t04:00:33.884][INFO][O.E.B. hootstrapchecks][dMy5nR5 non-loopback address, enforcing bootstrap checks [2019-03-06T04:00:36.995][INFO][O.E.c.s.asterservice][dMy5nR5] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {dMy5nR5} {dMy5nR5fThaBb - Q2T0txdA} {ldgTZ1XZSfOpda9uP4treA} {192.168.136.142} {9300} 192.168.136.142: {ml. Machine_memory = 10282 10688, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} [the 2019-03-06 T04:00:37, 003] [INFO] [O.E.C.S.C lusterApplierService] [dMy5nR5] new_master {dMy5nR5} {dMy5nR5fThaBb - Q2T0txdA} {ldgTZ1XZSfOpda9uP4treA} {192.168.136.142} {9300} 192.168.136.142: {ml. Machine_memory = 10282 10688, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {dMy5nR5} {dMy5nR5fThaBb - Q2T0txdA} {ldgTZ1XZSfOpda9uP4treA} {192.168.136.142} {9300} 192.168.136.142: {ml. Machine_memory = 10282 10688, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source[zen-disco-elected-as-master ([0] nodes joined)[, [2019-03-06]]]) T04:00:37, 058] [INFO] [O.E.X.S.T.N.S ecurityNetty4HttpServerTransport] [dMy5nR5] publish_address {9200} 192.168.136.142:. Bound_addresses {[::]:9200} [2019-03-06T04:00:37.058][INFO][O.E.n.ode][dMy5nR5] Started [2019-03-06T04:00:37.177][INFO][o.w.a.d.itor] try load config from / home/esyonghu/elasticsearch - 6.4.0 / config/analysis - ik/IKAnalyzer CFG. XML [the 2019-03-06 T04:00:37, 179] [INFO ] [O.W.A.D.M onitor] try load config the from/home/esyonghu/elasticsearch - 6.4.0 / plugins/ik/config/IKAnalyzer CFG. XML [the 2019-03-06 T04:00:37, 888] [INFO] [O.E.M.J.J vmGcMonitorService] [dMy5nR5] [gc] [4] overhead, spent 486 ms collectinginThe last 1.2 [s] [the 2019-03-06 T04:00:38, 435] [WARN] [O.E.X.S.A.S.M.N ativeRoleMappingStore] [dMy5nR5] Failed to clear the cacheforRealms [[]] [2019-03-06T04:00:38.469][INFO][O.E.L.lens][dMy5nR5] License [c91CAe39-79D7-4A0E-b40B-b1918a45F80C] mode [trial] -valid [2019-03-06T04:00:38.477][INFO][o.e.g.goatewayService] [dMy5nR5] Recovered [5] Indices into Cluster_state [2019-03-06T04:00:38,902 ][o.e.x.s.a.s.m.NativeRoleMappingStore] [dMy5nR5] Failed to clear cacheforRealms [[]] [the 2019-03-06 T04:00:39, 106] [INFO] [O.E.C.R.A.A llocationService] [dMy5nR5] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[mynote2][2]] ...] ).Copy the code

2. Now usefree -mTo view memory usage, you can see that 1.7GB has been used by swap

[esyonghu@localhost elasticsearch-6.4.0]$ free -m
             total       used       free     shared    buffers     cached
Mem:           980        916         64          0          3         33
-/+ buffers/cache:        880        100
Swap:         4095       1735       2360
[esyonghu@localhost elasticsearch-6.4.0]$
Copy the code