In my previous article “Elasticsearch: Setting Up Elastic Account Security” I covered in detail how to configure security for Elasticsearch. There we used the elasticSearch-setup-passwords command to configure the user name and password. This method requires us to manually create and input passwords, or commands to generate random passwords for us. The downside of this approach is that it is not easy to deploy automatically. If we had 10,000 servers, we would need to do 10,000 operations to get everything done. In today’s article, we will use elasticSearch -keystore to set the password, and use API methods to set other built-in user passwords. The advantage of this approach is that you can use scripts or other automated deployment tools such as Ansible to do this.
In today’s tutorial, I will use an Ubuntu OS 20.04 machine for installation and deployment.
Install Elasticsearch on your Ubuntu machine
If you have never installed Elasticsearch on your machine, you can refer to my previous article “How to install Elastic Stack on AWS Step by step”. We can follow the following steps to install.
Installing Elasticsearch on Ubuntu is easy. We will enable the Elasticsearch repository, import the repository GPG key, and then install the Elasticsearch server. The Elasticsearch package comes with a bundled version of OpenJDK, so you don’t have to install Java. First, update the package index and install the dependencies needed to add the new HTTPS repository:
sudo apt update
sudo apt install apt-transport-https ca-certificates wget
Copy the codeGPG key imported into warehouse:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Copy the codeThe command above should print OK, indicating that the key has been successfully imported and that packages from this repository will be treated as trusted packages.
Next, add the Elasticsearch repository to your system by issuing the following command:
sudo sh -c 'echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list'
Copy the codeAfter enabling the repository, install Elasticsearch by entering the following command:
sudo apt update
sudo apt install elasticsearch
Copy the codeAfter the installation is complete, the Elasticsearch service will not automatically start. To start the service and enable it to run, do the following:
sudo service elasticsearch start
Copy the codeTo check whether the Elasticsearch service has been successfully started, run the following command:
service elasticsearch status
Copy the codeIf you see something like this:
$elasticSearch status - ElasticSearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor> Active: active (running) since Tue 2021-01-26 08:17:24 CST; 3h 45min ago Docs: https://www.elastic.co Main PID: 1809 (Java) Tasks: 90 (LIMIT: 18985) Memory: 1.5g CGroup: / system. Slice/elasticsearch service ├ ─ 1809 / usr/share/elasticsearch/JDK/bin/Java - Xshare: auto - Des.net > └ ─ 2448 The/usr/share/elasticsearch/modules/x - pack - ml/platform/Linux - January 26 x > 08:17:12 liuxgu systemd [1] : Starting Elasticsearch... Liuxgu Systemd [1]: Started Elasticsearch 1月 26 08:17:24 Liuxgu Systemd [1]: Started ElasticsearchCopy the codeThe Elasticsearch service is enabled. To check Elasticsearch, run the following command:
curl -X GET "localhost:9200/"
Copy the code$ curl -X GET "localhost:9200/" { "name" : "elk-1", "cluster_name" : "demo-elk", "cluster_uuid" : "RvdA4ZxgTqOEp6O28CT05w ", "version" : {"number" : "7.10.2", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9", "build_date" : "2021-01-13T00:42:12.435326z ", "build_snapshot" : false, "lucene_version" : "Minimum_index_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1"}, "tagline" : "You Know, for Search"}Copy the codeIf you can see the output above, Elasticsearch is working properly. You can check Elasticsearch run logs by using the following command:
sudo journalctl -u elasticsearch
Copy the code$ sudo journalctl -u elasticsearch -- Logs begin at Tue 2020-10-27 19:19:12 CST, End at Tue 2021-01-26 12:05:55 CS> 1月 25 18:40:24 Liuxgu Systemd [1]: Starting Elasticsearch... Liuxgu Systemd [1]: Started Elasticsearch. -- Reboot -- liuxgu Systemd [1]: Starting Elasticsearch... Liuxgu Systemd [1]: Started Elasticsearch 1月 26 08:17:24 Liuxgu Systemd [1]: Started ElasticsearchCopy the codeIt shows that everything is working properly.
Elasticsearch keystore is introduced
You can configure security for Elasticsearch as described in the previous article “Elasticsearch: Setting Elastic Account security”. After the configuration is complete, you must restart the ElasticSearch service. In today’s article, we’ll use a different approach. This approach will make it easy to use scripts or tools like Ansible for automated deployment in the future. First let’s take a look at two important categories:
Es_home directory
/usr/share/elasticsearch
Copy the codeIf you take a closer look at Elasticsearch’s installation directory, you will see that Elasticsearch is installed in the above directory:
$ pwd
/usr/share/elasticsearch
liuxg@liuxgu:/usr/share/elasticsearch$ ls
NOTICE.txt README.asciidoc bin jdk lib modules plugins
Copy the codeEs_conf_dir directory
/etc/elasticsearch
Copy the codeThis is a configuration directory for Elasticsearch. It has root permission. We must use the following method to enter:
$ su
Password:
root@liuxgu:/home/liuxg# cd /etc/elasticsearch
root@liuxgu:/etc/elasticsearch# ls
elasticsearch.keystore jvm.options.d roles.yml
elasticsearch.yml log4j2.properties users
jvm.options role_mapping.yml users_roles
Copy the codeAbove you can see the Elasticsearch allocation file. Here we can see the elasticSearch.keystore file. This is used to store some key/value data that you don’t want others to know.
Familiar with elasticsearch – keystore
Get familiar with ElasticSearch -keystore in this section. In /usr/share/elasticsearch, type the following command:
sudo ./bin/elasticsearch-keystore -help
Copy the codeThe command above shows:
$ sudo ./bin/elasticsearch-keystore -help A tool for managing settings stored in the elasticsearch keystore Commands -------- create - Creates a new elasticsearch keystore list - List entries in the keystore add - Add a string settings to the keystore add-file - Add a file setting to the keystore remove - Remove settings from the keystore upgrade - Upgrade the keystore format passwd - Changes the password of a keystore has-passwd - Succeeds if the keystore exists and is password-protected, fails with exit code 1 otherwise. Non-option arguments: command Option Description ------ ----------- -E <KeyValuePair> Configure a setting -h, --help Show help -s, --silent Show minimal output -v, --verbose Show verbose outputCopy the codeFor example, we use the following command to check whether the keystore contains password:
$ sudo ./bin/elasticsearch-keystore has-passwd
ERROR: Keystore is not password-protected
Copy the codeIt shows that our keystore is not currently password protected. We can use the following command to set the password:
$ sudo ./bin/elasticsearch-keystore passwd
Enter new password for the elasticsearch keystore (empty for no password):
Enter same password again:
Elasticsearch keystore password changed successfully.
Copy the codeOn it, we don’t type anything, so we don’t have any passwords. We will set this password in the following exercise. Next, let’s see how to add an item to the keystore:
$ sudo ./bin/elasticsearch-keystore add user.name
Enter value for user.name:
Copy the codeUp here, we type anything. Then we can use the following command to check:
$ sudo ./bin/elasticsearch-keystore list
keystore.seed
user.name
Copy the codeAbove, we can see that a user. Name item is listed. This is what we just created. We can delete it by using the following command:
$ sudo ./bin/elasticsearch-keystore remove user.name
liuxg@liuxgu:/usr/share/elasticsearch$ sudo ./bin/elasticsearch-keystore list
keystore.seed
Copy the codeAfter deleting it, we’ll do a list to show it. We find that the user.name created just now is missing.
Elastic bootstrap password
When installing Elasticsearch, if the Elastic user does not already have a password, it will use the default bootstrap. Bootstrap is a temporary password that allows you to run a tool that sets passwords for all built-in users.
By default, the bootstrap password comes from the randomized keystore.seed setting that was added to the keystore during installation. You do not need to know or change this bootstrap password. However, if you define the bootstrap.password setting in your keystore, this value will be used. For more information about interacting with keystores, see Security Settings.
Note: After setting passwords for built-in users (especially elastic users), bootstrap passwords will no longer be used.
Configure security for Elasticsearch
In order to secure Elasticsearch, we must modify the Elasticsearch configuration file. The Elasticsearch. Yml file is found in the Elasticsearch configuration directory. We add the following line to the file:
/etc/elasticsearch/elasticsearch.yml
xpack.security.enabled: true
Copy the codeYou need to modify the above file with root permission. We can restart elasticSearch service:
sudo service elasticsearch restart
Copy the codeIf we go to http://localhost:9200 at this time, we will not be able to access Elasticsearch. This is because we now have security in place. We need to use a username and password to access it.
$ curl -X GET “localhost:9200/”
{“error”:{“root_cause”:[{“type”:”security_exception”,”reason”:”missing authentication credentials for REST request [/]”,”header”:{“WWW-Authenticate”:”Basic realm=\”security\” charset=\”UTF-8\””}}],”type”:”security_exception”,”reason”:”missing authentication credentials for REST request [/]”,”header”:{“WWW-Authenticate”:”Basic realm=\”security\” charset=\”UTF-8\””}},”status”:401}
We next install the following steps to configure the secure user:
Create elasticSearch keystore
$ pwd
/usr/share/elasticsearch
liuxg@liuxgu:/usr/share/elasticsearch$ sudo ./bin/elasticsearch-keystore create
[sudo] password for liuxg:
An elasticsearch keystore already exists. Overwrite? [y/N]y
Created elasticsearch keystore in /etc/elasticsearch/elasticsearch.keystore
Copy the code2) Set ElasticSearch keystore permissions
sudo chown root:elasticsearch /etc/elasticsearch/elasticsearch.keystore
sudo chmod 0660 /etc/elasticsearch/elasticsearch.keystore
Copy the codeUnder the default installation, the/etc/elasticsearch/elasticsearch keystore attribute is what we need. We can skip this step.
Keystore -rw-rw---- 1 root elasticSearch 199 1月 26 12:58 elasticSearchCopy the code3) Run the following command to check whether the Elasticsearck keystore is set
$ sudo ./bin/elasticsearch-keystore list
keystore.seed
Copy the code4) Configure the bootstrap password
We can refer to the official document “Built-in Users” for configuration. We need to configure the password for bootstrap. We can use the following command to get some help:
$ sudo ./bin/elasticsearch-keystore add -help
Add a string settings to the keystore
Non-option arguments:
setting names
Option Description
------ -----------
-E <KeyValuePair> Configure a setting
-f, --force Overwrite existing setting without prompting, creating
keystore if necessary
-h, --help Show help
-s, --silent Show minimal output
-v, --verbose Show verbose output
-x, --stdin Read setting values from stdin
Copy the codeWe can use the -x option to read the password from stdin and configure:
$ pwd
/usr/share/elasticsearch
liuxg@liuxgu:/usr/share/elasticsearch$ echo "demopassword" | sudo ./bin/elasticsearch-keystore add -x "bootstrap.password"
Copy the codeAbove, we set the password demopassword for bootstrap. After the above setup, we can use the following command to check the ElasticSearch keystore:
$ sudo ./bin/elasticsearch-keystore list
bootstrap.password
keystore.seed
Copy the codeWe can see an item called bootstrap.password.
5) Restart elasticSearch
To restart the ElasticSearch service, use the following command:
sudo service elasticsearch restart
Copy the codeAfter the above restart, we can use the following method to check:
curl -u elastic:demopassword localhost:9200
Copy the codeAbove, we used the super account elastic and the password demopassWord we set earlier. The command above shows the result:
$ curl -u elastic:demopassword localhost:9200 { "name" : "elk-1", "cluster_name" : "demo-elk", "cluster_uuid" : "RvdA4ZxgTqOEp6O28CT05w ", "version" : {"number" : "7.10.2", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9", "build_date" : "2021-01-13T00:42:12.435326z ", "build_snapshot" : false, "lucene_version" : "Minimum_index_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1"}, "tagline" : "You Know, for Search"}Copy the codeObviously this is the right setting.
6) Create accounts for built-in users
For the sake of explanation, we will use the following account and password:
# A built-in superuser.
elastic_username: elastic
elastic_password: goodwitch
# The user Kibana uses to connect and communicate with Elasticsearch.
kibana_username: kibana
kibana_password: badsanta
# The user Logstash uses when storing monitoring information in Elasticsearch.
logstash_system_username: logstash_system
logstash_system_password: dragonprince
# The user the Beats use when storing monitoring information in Elasticsearch.
beats_system_username: beats_system
beats_system_password: avatar
# The user the APM server uses when storing monitoring information in Elasticsearch.
apm_system_username: apm_system
apm_system_password: mashaandthebear
# The user Metricbeat uses when collecting and storing monitoring information in Elasticsearch. It has the remote_monitoring_agent and remote_monitoring_collector built-in roles.
remote_monitoring_user_username: remote_monitoring_user
remote_monitoring_user_password: gossipgir
Copy the codeWe can create usernames and passwords directly using the REST API, for example:
curl -u elastic:demopassword -XPOST “http://localhost:9200/_security/user/elastic/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “password”}’
For example, above, we used the API to change the password of an Elastic user directly. With this change, our superuser Elastic’s password will now be password instead of DemopassWord:
$ curl -u elastic:demopassword localhost:9200
{“error”:{“root_cause”:[{“type”:”security_exception”,”reason”:”unable to authenticate user [elastic] for REST request [/]”,”header”:{“WWW-Authenticate”:”Basic realm=\”security\” charset=\”UTF-8\””}}],”type”:”security_exception”,”reason”:”unable to authenticate user [elastic] for REST request [/]”,”header”:{“WWW-Authenticate”:”Basic realm=\”security\” charset=\”UTF-8\””}},”status”:401}
Instead, this is the way:
$ curl -u elastic:password localhost:9200 { "name" : "elk-1", "cluster_name" : "demo-elk", "cluster_uuid" : "RvdA4ZxgTqOEp6O28CT05w ", "version" : {"number" : "7.10.2", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9", "build_date" : "2021-01-13T00:42:12.435326z ", "build_snapshot" : false, "lucene_version" : "Minimum_index_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1"}, "tagline" : "You Know, for Search"}Copy the codeWe can install the same method as above to create accounts for other built-in accounts:
- Kibaba
curl -u elastic:password -XPOST “http://localhost:9200/_security/user/kibana/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “badsanta”}’
- Logstash
curl -u elastic:password -XPOST “http://localhost:9200/_security/user/logstash_system/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “dragonprince”}’
- Beats
curl -u elastic:password -XPOST “http://localhost:9200/_security/user/beats_system/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “avatar”}’
- APM
curl -u elastic:password -XPOST “http://localhost:9200/_security/user/apm_system/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “mashaandthebear”}’
- Remote monitoring
curl -u elastic:password -XPOST “http://localhost:9200/_security/user/remote_monitoring_user/_password” -H ‘Content-Type: application/json’ -d'{ “password”: “gossipgirl”}’
So we’ve created accounts for all of our built-in users.
After setting this up and accessing the Ubuntu machine from another machine, we need to use elastic/password to access it:
Enter elastic and password password:
conclusion
In today’s exercise, we configured security for Elasticsearch. We created access to superuser Elastic by setting the bootstrap password through the ElasticSearch keystore. We then use this superuser and use the REST API to create other built-in users. This method is different from the one described in our previous article “Elasticsearch: Setting Elastic Account security”. But this approach makes it easy to deploy on a large scale using scripts or tools like Ansible without having to interact. In the next article, I’ll show you how to automate Elasticsearch deployment using Ansible. Stay tuned!
Reference:
【 1 】 linuxize.com/post/how-to…