0 background

In the production environment, we have set up two sets of ES cluster successively. One set is used to collect system logs, including K8S component logs, gateway logs, service POD logs, etc. The other is used for the storage of business system information, such as buried information, business logs, etc. Now, we want to unify the two sets of cluster management, using a unified Kibana panel to do external data display. Searching for solutions on the Internet, I saw the remote clustering function of ES, which is recorded here.

Elasticsearch introduced the cross-cluster Search (CCS cross-cluster Search) function in version 5.3 to replace the Tribe Node to be deprecated. Similar to Tribe Node, Cross Cluster Search is used to Search data across clusters. Cross-cluster search enables you to run a single search request against one or more remote clusters. For example, you can use cross-cluster search to filter and analyze log data stored in clusters in different data centers.

1 Configure the remote cluster

ES provides two remote cluster configuration schemes. One is directly configured in the ES configuration file ElasticSearch. yml, and the other is configured using the API.

1.1 Configuration File Configuration Mode

Modify elasticSearch. yml and add the following information

Search: Remote: Cluster01: Seeds: 192.168.10.101:9300 seeds: 192.168.10.102:9300 seeds: 192.168.10.103:9300 transport.compress: true Skip_unavailable: trueCopy the code
  • cluster01: Specifies the cluster name
  • seeds: Node list of the cluster. You can configure one or more nodes
  • transport.ping_schedule: Interval for ping to check the connection status
  • skip_unavailable: Whether to skip unavailable clusters when searching across clusters

1.2 API Configuration Mode

Use the Cluster Settings API

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster01": {
          "skip_unavailable": false,
          "mode": "sniff",
          "proxy_address": null,
          "proxy_socket_connections": null,
          "server_name": null,
          "seeds": [
            "cluster01_node01:9300",
            "cluster01_node02:9300",
            "cluster01_node03:9300"
          ],
          "node_connections": 3
        }
      }
    }
  }
}
Copy the code

The configuration parameters are the same as the preceding

I prefer to use the API to set up the remote cluster, which makes it easier to modify the remote cluster.

1.3 Viewing the Status of the Remote Cluster

Use the GET _remote/info request to view

{ "cluster01" : { "connected" : true, "mode" : "sniff", "seeds" : ["192.168.10.101:9300", "192.168.10.102:9300", "192.168.10.103:9300"], "num_nodes_connected" : 3, "max_connections_per_cluster" : 3, "initial_connect_timeout" : "30s", "skip_unavailable" : false } }Copy the code

1.4 Deleting a Remote Cluster

If you set the seeds incorrectly or don’t want to use it anymore, you can delete the remote cluster, essentially setting the seeds to empty.

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster01": {
          "seeds": null 
        }
      }
    }
  }
}
Copy the code

1.5 Using Kibana to manage remote clusters

If You use Kibana, setting up a remote cluster is much easier, just on the page.

Open Stack Management, locate the remote cluster or Remote Cluster in the left directory, and click Add Remote Cluster on the displayed page

Enter the cluster name and node information

Can be saved

2 Use the remote cluster to search

Remote cluster permission Settings

  1. Create a role on the remote cluster with the same name as the local cluster
  2. The role on the remote cluster to be assigned to the corresponding indexreadwithread_cross_clusterPermission, otherwise the connection will be rejected when the local cluster accesses the index.

Querying a Remote Cluster

To query the index of a remote cluster, specify the cluster name

GET /cluster_name:index/_search
Copy the code

Query multiple clusters simultaneously

GET /cluster_name:index,cluster_name:index/_search
Copy the code

Query all clusters simultaneously

GET */index/_search
Copy the code

Create index schema in Kibana

When creating the index schema in Kibana, you also specify the cluster name

cluster_name:index*
Copy the code

Then use the same index pattern as the local cluster.