Small knowledge, big challenge! This paper is participating in theEssentials for programmers”Creative activities

The output to Redis

Configuration of the sample

input { stdin {} }
output {
    redis {
        data_type => "channel"
        key => "logstash-chan-%{+yyyy.MM.dd}"
    }
}
Copy the code

Usage

We will continue with the Redis – CLI command line to demonstrate the essence of the Outputs/Redis plugin.

basical use case

Run the logstash process and then the other terminal starts the redis-cli command. After entering the Redis command (“SUBSCRIBE logstash-chan-2014.08.08”) to SUBSCRIBE to the specified channel, you will first see a message indicating that the subscription was successful. As follows:

# redis-cli 127.0.0.1:6379> SUBSCRIBE Logstash -chan-2014.08.08 Reading Messages... (press Ctrl-c to quit) 1) "subscribe" 2) "logstash-chan-2014.08.08" 3) (integer) 1Copy the code

Okay, so in the terminal where logstash is running, type in the “Hello World” string. Switch back to the redis-CLI terminal and you will find that a message has been automatically output:

1) "message" 2) "logstash-chan-2014.08.08" 3) "{\"message ":\"hello World \ ", \ "@ version \" : \ "1 \", \ "@ timestamp \", \ "the T16 2014-08-08:" the 865 z \ "and \" host \ ", \ "raochenlindeMacBook - Air. Local \"}"Copy the code

Does it look familiar? This string of characters is the inputs/redis section.

Elsif outputs/redis and inputs/redis

In fact, this is how we use the Redis server as the broker role in the LogSTassh architecture.

Let’s run the logstash process with different configurations from these two sections on both terminals, without running the redis-CLI command this time. Input “Hello world” in inputs/redis to output data automatically.

notification use case

We can also use other programs to subscribe to the Redis channel, and we can write any other logic in the program. You can see how the Output/Juggernaut plug-in works. The Juggernaut is built on the Redis server and socket. IO framework. With this, Logstash can push alerts directly to socket.io browsers such as WebKit.

scaling

As with LogStash::Inputs::Redis, there is a way to set it to list. Send it to the Redis server using the RPUSH command, and the effect is exactly the same as shown earlier. Including the adjustable parameter batch_event, also described in the previous section. I will not repeat the examples here.

The output to Statsd

Statsd is the first Flickr company in 2008 written with Perl for Graphite, Datadog and other monitoring data back-end storage development of front-end network applications, in 2011 Etsy company with NodeJS reconstruction. Used for receiving, writing, reading and aggregating time series data, including time values and cumulative values.

Configuration of the sample

output {
    statsd {
        host => "statsdserver.domain.com"
        namespace => "logstash"
        sender => "%{host}"
        increment => ["httpd.response.%{status}"]
    }
}
Copy the code

explain

Graphite stores monitoring data in a tree-like structure, so does STATSD. Therefore, the key of the data sent to Statsd must be of the form “first.second.tree.four”. Outputs/STATSD will concatenate three configuration parameters:

    namespace.sender.metric
Copy the code

Namespace and Sender are set directly, and metric is divided into several parameters that can be set separately. Statsd supports the following metric types:

The metric type

• increment

Example syntax: Increment => [“nginx.status.%{status}”]

• decrement

Syntax is the same as increment.

• count

Example syntax: count => {“nginx.bytes” => “%{bytes}”}

• gauge

Syntax is the same as count.

• set

Syntax is the same as count.

• timing

Syntax is the same as count.

For detailed descriptions of these metric types, please read the STATSD documentation: github.com/etsy/statsd… .

Recommended reading

• Etsy launches nodeJS version of Statsd’s blog: Measure Anything, Measure Everything
• Flickr publishes The Counting & Timing blog statsd

Call the police to Nagios

There are two output plug-ins in Logstash that are related to Nagios. The outputs/nagios plugin sends data to the native nagios.cmd pipeline command file, The outputs/ NagiOS_Nsca plugin calls send_nsca to send data to the Nagios server (remote or local) in NSCA format.

Nagios.Cmd

CMD is the core component of the Nagios server. Nagios event processing and internal and external interactions are done through this pipe file.

In CMD mode, you need to ensure that the Logstash event sent conforms to the nagios event format. The nagiOS_HOST and nagiOS_Service fields must be prepared in the Filter phase. In addition, nagiOS_ANNOTATION and NagiOS_Level fields are automatically converted to Nagios event information if they are also prepared in the Filter phase.

filter {
    if [message] =~ /err/ {
        mutate {
            add_tag => "nagios"
            rename => ["host", "nagios_host"]
            replace => ["nagios_service", "logstash_check_%{type}"]
        }
    }
}
output {
    if "nagios" in [tags] {
        nagios { }
    }
}
Copy the code

If you do not intend to provide nagiOS_Level at the Filter stage, you can also configure it through parameters in the plug-in.

Nagios_level is the value returned when we check the data with the Nagios Plugin. The value range and meanings are as follows:

• “0” indicates “OK”, indicating that the service is normal.
• “1”, which stands for “WARNNING”, is a service warning used in the nagios plugin command-wParameter Sets the threshold.
• “2” stands for “CRITICAL”, which is used in the Nagios plugin command-cParameter Sets the threshold.
• “3” stands for “UNKNOWN”, which usually occurs in the case of timeout.

By default, the plugin sends an alarm to the Nagios server with a “CRITICAL” rating.

The location of the nagios. CMD file can be set using the command_file parameter. The default location is “/ var/lib/nagios3 / rw/nagios. CMD”.

For detailed protocols for interacting with nagios.cmd, read Using External Commands in Nagios, an excerpt from the Learning Nagios 3.0 book.

NSCA

NSCA is a standard Nagios distributed extension protocol. The send_nsca process distributed across the machines actively pushes monitoring data to the NSCA process of the remote Nagios server.

When the Logstash and Nagios servers are not running on the same host, NSCA is the only way to send alerts — send_nsca must also be installed on the Logstash server.

Several attributes required by nagios events are described in the previous section. When using the plugin, however, you don’t need to prepare it in advance. Instead, you can define parameters inside the plugin:

output {
    nagios_nsca {
        nagios_host => "%{host}"
        nagios_service => "logstash_check_%{type}"
        nagios_status => "2"
        message_format => "%{@timestamp}: %{message}"
        host => "nagiosserver.domain.com"
    }
}
Copy the code

Note that the host and nagiOS_host parameters are used to set the address of the Nagios server and the address of the server in question in the alarm message.

Using NSClient++ from nagios with NSCA for details on NSCA principles, architecture, and configuration.

Recommended reading

In addition to Nagios, Logstash can also send information to other common monitoring systems. The approach is similar to nagios:

• outputs/gangliaThe plugin sends gmetric data to the local/remote end over UDPgmondorgmetad
• outputs/zabbixThe plug-in calls the nativezabbix_senderCommand sent

Sending emails (Email)

Configuration of the sample

output { email { to => "[email protected],[email protected]" cc => "[email protected]" via => "smtp" subject => "Warning:  %{title}" options => { smtpIporHost => "localhost", port => 25, domain => 'localhost.localdomain', userName => nil, password => nil, authenticationType => nil, # (plain, login and cram_md5) starttls => true } htmlbody => "" body => "" attachments => ["/path/to/filename"] } }Copy the code

explain

Outputs /email plugin supports BOTH SMTP protocol and sendmail via parameter setting. In SMTP mode, many options parameters can be configured. Sendmail can only be done using the local sendmail service — the documentation describes the sendmail configuration parameters supported by the Mail library, but the actual code does not handle this, so don’t be confused…

Call command Execution (Exec)

The Outputs /exec plugin is also very simple to use, as shown below, passing the logstash content as a parameter to the command. This will trigger the execution of the command every time an event arrives at the plug-in.

output {
    exec {
        command => "sendsms.pl \"%{message}\" -t %{user}"
    }
}
Copy the code

One thing to note. This is done by restarting the command and exiting each time. The process itself is relatively slow (program loading, network connection and so on have a certain amount of time consumption). It is best used for a small number of information-processing scenarios, such as other ways of alerting that nagios does not apply. The example is sending a message via SMS.