Some of the following configurations need to be adjusted when merging from a standalone deployment server to a cluster.

The Master service

If you have a standalone server deployment that already exists and runs independently, such as the server deployed in the page single-server Deployment Examples, The following example will help you merge Coordinator and Overlord into one process

The conf/druid/cluster/master/coordinator – overlord in the following example, show such as how to merge the coordinator and overlord process at the same time.

You can be independent from already deployed on the server copy existing coordinator – overlord configuration files, and deploy to the conf/druid/cluster/master/coordinator – overlord.

The Data service

Suppose we are going to merge from a separate server with 32 cpus and 256GB of memory. In the old deployment, the following configuration was for Historicals and MiddleManagers processes:

Historical (Standalone server deployment)

druid.processing.buffer.sizeBytes=500000000
druid.processing.numMergeBuffers=8
druid.processing.numThreads=31
Copy the code

MiddleManager (Standalone server Deployment)

druid.worker.capacity=8
druid.indexer.fork.property.druid.processing.numMergeBuffers=2
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000
druid.indexer.fork.property.druid.processing.numThreads=1
Copy the code

In a clustered deployment environment, we have the option of running the two services with two servers configured with 16 CPUS and 128GB of RAM. We will configure it as follows:

Historical

  • druid.processing.numThreads: configure-based new hardware environment, set to(num_cores - 1)
  • druid.processing.numMergeBuffers: Divide the number of independent servers by a split factor
  • druid.processing.buffer.sizeBytes: Stay the same

MiddleManager:

  • druid.worker.capacity: Divide the number of independent servers by a split factor
  • druid.indexer.fork.property.druid.processing.numMergeBuffers: Stay the same
  • druid.indexer.fork.property.druid.processing.buffer.sizeBytes: Stay the same
  • druid.indexer.fork.property.druid.processing.numThreads: Stay the same

The configuration results are as follows:

Cluster Historical (using 2 data servers)

 druid.processing.buffer.sizeBytes=500000000
 druid.processing.numMergeBuffers=8
 druid.processing.numThreads=31
Copy the code

Cluster MiddleManager (with 2 data servers)

druid.worker.capacity=4
druid.indexer.fork.property.druid.processing.numMergeBuffers=2
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000
druid.indexer.fork.property.druid.processing.numThreads=1
Copy the code

The Query service

You can copy the configuration files that already exist in standalone server deployment to conf/druid/cluster/query to complete the deployment. If the hardware configuration of the new server is relative to that of the standalone server, no changes are required for the new deployment.

Refreshing deployment

If you use the following server configuration environment as an example:

  • 1 Master server (m5.2 xlarge)
  • 2 Data servers (i3.4 xlarge)
  • 1 the Query server (m5.2 xlarge)

The configuration files in the conf/druid/cluster folder have been optimized for the above hardware environment, so you don’t need to change the configuration for basic use.

If you choose to use different hardware, the Basic Cluster Tuning Guide can help you make some choices about your hardware configuration.

www.ossez.com/t/druid/136…