This article describes installing Kafka for Linux.

1. Introduction of Kafka

Kafka is also a project of the Open Source and Apache Open Source Foundations, written in Scala and Java. Kafka is a high throughput distributed publish subscribe messaging system.

Baidu Encyclopedia is introduced like this:

Kafka is an open source stream processing platform developed by the Apache Software Foundation and written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that processes all action flow data in consumer-scale websites. This action (web browsing, searching and other user actions) is a key factor in many social functions on the modern web. This data is usually addressed by processing logs and log aggregation due to throughput requirements. This is a viable solution for logging data and offline analysis systems like Hadoop, but with limitations that require real-time processing. Kafka is designed to unify online and offline message processing through Hadoop’s parallel loading mechanism, and to provide real-time messaging across clusters.

Kafka’s official website is kafka.apache.org/.

2. Preparation

2.1 Installation Environment

In this document, Kafka 2.10 is installed. CentOS 7.2 is used in Linux. Zookeeper is installed in /usr/local/software/.

2.2 Preparing Tools

1.Java

Kafka relies on the Java environment to run, so you need to install the Java environment on your Linux system.

2. Installation of Kafka

The steps can be roughly divided into the following steps:

2.1 download the Kafka

Wget HTTP: / / http://labfile.oss.aliyuncs.com/courses/859/kafka_2.10-0.10.2.1.tgzCopy the code

2.2 unzip the Kafka

The tar - ZXVF kafka_2. 10-0.10.2.1. TGZCopy the code

2.3 Going to the Configuration File Directory

cd /usr/local/ software/kafka_2. 10-0.10.2.1 / configCopy the code

2.4 Modifying the configuration file server.properties and add the following information:

broker.id=0
port=9092 # portThe host name = 172.30.0.9Change the server IP address to your own server IP address
log.dirs=/usr/local/logs/kafka # log path, the directory created above
zookeeper.connect=localhost:2181 Localhost :2181
Copy the code

2.5 Writing startup Scripts

vi kafkastart.sh
Copy the code
# start the zookeeper
/usr/local/ software/kafka_2. 10-0.10.2.1 / bin/zookeeper - server - start. Sh/usr /local/ software/kafka_2. 10-0.10.2.1 / config/zookeeper. The properties &# Wait 3 seconds before executing
sleep 3 
# start kafka
/usr/local/ software/kafka_2. 10-0.10.2.1 / bin/kafka - server - start. Sh/usr /local/ software/kafka_2. 10-0.10.2.1 / config/server properties &Copy the code

2.6 Writing a Shutdown Script

vi kafkastop.sh
Copy the code
# close the zookeeper
/usr/local/ software/kafka_2. 10-0.10.2.1 / bin/zookeeper - server - stop. Sh/usr /local/ software/kafka_2. 10-0.10.2.1 / config/zookeeper. The properties &# Wait 3 seconds before executing
sleep 3 
Close # kafka
/usr/local/ software/kafka_2. 10-0.10.2.1 / bin/kafka - server - stop. Sh/usr /local/ software/kafka_2. 10-0.10.2.1 / config/server properties &Copy the code

2.7 Starting the Script and Disabling The Script Permission

chmod 777 kafkastart.sh
Copy the code
chmod 777 kafkastop.sh
Copy the code

3. Installation problems

The installation of Kafka didn’t run into any major problems, and the input went through.

4. Install the visual management page

Kafka’s official website: github.com/linkedin/ka… , is an executable jar (kafkaoffsetMonitor-assembly-0.2.1.jar).

4.1 Writing startup Scripts

Java -xms128m -xmx128m -xSS1024K -xx :PermSize=128m -xx :MaxPermSize=128m -cp kafkaoffsetMonitor-assembly-0.2.1.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk localhost:2181 --port 8787 --refresh 10.seconds --retain 7.days 1>/usr/local/software/kafka-monitor/stdout.log 2>/usr/local/software/kafka-monitor/stderr.log &
Copy the code

5. To summarize

The above installation is through my personal test available, the installation process is relatively simple.