Preface: If you want to do a good job, you must first use powerful tools. Today, I would like to introduce ten fancy sharp tools of HBase Shell to you, which you can try to use in daily operation and maintenance work.

1. Interactive mode

This is the Shell command line approach we most commonly use.

$ hbase shell

hbase(main):001:0> listCopy the code

2. Non-interactive mode

$echo "describe" test1 "|" hbase shell - n # $echo results output to a file "describe" test1 "| hbase shell -n > TMP, log # $not printing results echo "describe 'test'" | hbase shell -n > /dev/null 2>&1Copy the code

Compare with interactive mode

If we want to know if HBase Shell commands have been successfully executed, we must use non-interactive mode. Interactive mode always returns 0 after executing a command. When the command fails to execute, non-interactive mode returns a non-zero value. The following is an example:

$ echo "error cmd" | hbase shell > /dev/null 2>&1
$ echo $?
0

$ echo "error cmd" | hbase shell -n > /dev/null 2>&1
$ echo $?
1Copy the code

3. Use Ruby scripts

$ hbase org.jruby.Main PATH_TO_SCRIPTCopy the code

Take the get-active-master.rb script in the HBase bin directory as an example:

#! /usr/bin/env hbase-jruby include Java import org.apache.hadoop.hbase.HBaseConfiguration import org.apache.hadoop.hbase.ServerName import org.apache.hadoop.hbase.zookeeper.ZKUtil import org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher # disable debug/info logging on this script for clarity log_level = org.apache.log4j.Level::ERROR org.apache.log4j.Logger.getLogger('org.apache.hadoop.hbase').setLevel(log_level) org.apache.log4j.Logger.getLogger('org.apache.zookeeper').setLevel(log_level) config = HBaseConfiguration.create zk = ZooKeeperWatcher.new(config, 'get-active-master', nil) begin master_address = ZKUtil.getData(zk, zk.masterAddressZNode) if master_address puts ServerName.parseFrom(master_address).getHostname() else puts 'Master not running' end ensure zk.close() endCopy the code

Run the following command:

$ hbase org.jruby.Main get-active-master.rb
xxxxxxxx.xxx.com.cnCopy the code

4. Use the Bash script

Example 1:

#! /bin/bash echo "describe 'test:t1'" | hbase shell -n > tmp.log status=$? echo "The status was " $status if [ $status == 0 ]; then echo "The command succeeded" else echo "The command may have failed." fiCopy the code

Example 2:

#! /bin/bash arr=('test:t1' 'test:t2') for table in ${arr[@]} do echo \'$table\' hbase shell << EOF disable '$table' drop '$table' exit EOF doneCopy the code

Execute the script:

$ sh xxx.shCopy the code

5. Read the text file

Suppose my text file is sample_commands.txt with the following content:

create 'test', 'cf'
list 'test'
put 'test', 'row1', 'cf:a', 'value1'
put 'test', 'row2', 'cf:b', 'value2'
put 'test', 'row3', 'cf:c', 'value3'
put 'test', 'row4', 'cf:d', 'value4'
scan 'test'
get 'test', 'row1'
disable 'test'
enable 'test'
exitCopy the code

Run the following command:

$ hbase shell ./sample_commands.txtCopy the code

6. Execute Java code

Hbase (main) : 001-0 > import Java. Util. Date file: / usr/HDP / 2.6.5.0-292 / hbase/lib/ruby/engine - complete - 1.6.8. Jar! /builtin/javasupport/core_ext/object.rb:99 warning: already initialized constant Date => Java::JavaUtil::Date hbase(main):002:0> Date.new(1218920189000).toString() => "Sun Aug 17 04:56:29 CST 2008" hbase(main):004:0> import java.text.SimpleDateFormat => Java::JavaText::SimpleDateFormat hbase(main):005:0> import java.text.ParsePosition => Java::JavaText::ParsePosition hbase(main):006:0> SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29", ParsePosition.new(0)).getTime() => 1218891389000 hbase(main):003:0>Copy the code

7. Pass VM parameters

The VM options can be passed to the HBase Shell using the HBase_Shell_OPTS environment variable. You can set it in your environment, for example by editing ~/.bashrc, or by setting it as part of the HBase Shell command that starts it. The following example sets several garbage collector related variables that are used only for the lifetime of the VM running HBase Shell. The command should run on one line, but it is interrupted by the \ character for readability.

$ HBASE_SHELL_OPTS="-verbose:gc -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps \
  -XX:+PrintGCDetails -Xloggc:$HBASE_HOME/logs/gc-hbase.log" hbase shellCopy the code

8. Configure overwrite

In hbase-2.0.5/hbase-2.1.3/hbase-2.2.0/hbase-1.4.10/hbase-1.5.0, you can transfer or rewrite the hbase configuration specified in hbase-*. XML by passing the key/value prefixed with -d on the cli, as shown in the following:

$ hbase shell -Dhbase.zookeeper.quorum=ZK0.remote.cluster.example.org,ZK1.remote.cluster.example.org,ZK2.remote.cluster.example.org -Draining=false
...
hbase(main):001:0> @shell.hbase.configuration.get("hbase.zookeeper.quorum")
=> "ZK0.remote.cluster.example.org,ZK1.remote.cluster.example.org,ZK2.remote.cluster.example.org"
hbase(main):002:0> @shell.hbase.configuration.get("raining")
=> "false"Copy the code

9. Table variables

HBase 0.95 adds shell commands to provide JRuby-style object-oriented references to tables. Previously, all shell commands that acted on tables had a procedure style that always took the name of the table as an argument. HBase 0.95 introduced the ability to assign tables to JRuby variables. Table references can be used to perform data read and write operations, such as placing, scanning, and fetching, as well as administrative functions, such as disabling, deleting, and describing tables. For example, previously you always specified a table name:

Hbase (main):000:0> create 'test', 'F' 0 row(s) in 1.0970 seconds hbase(main):001:0> PUT 'test', 'rold', 'f', 'V' 0 row(s) in 0.0080 seconds hbase(main):002:0> Scan 'test' row COLUMN+CELL rold COLUMN = F :, timestamp=1378473207660, Value = V 1 row(s) in 0.0130 seconds hbase(main):004:0> disable 'test' 0 row(s) in 14.8700 seconds hbase(main):005:0> DROP 'test' 0 row(s) in 23.1670 seconds hbase(main):006:0>Copy the code

You can now assign a table to a variable and use the result in JRuby shell code.

Hbase (main):007 > t = create 'test', 'f' 0 row(s) in 1.0970 seconds => hbase ::Table -t hbase(main):008 > t.put 'r', 'F ', 'v' 0 row(s) in 0.0640 seconds hbase(main):009 > t.can row COLUMN+CELL R COLUMN = F :, timestamp=1331865816290, Value = V 1 row(s) in 0.0110 seconds hbase(main):038:0> t.disable 0 Row (s) in 6.2350 seconds hbase(main):039:0> t.drop 0 The row (s) in 0.2340 secondsCopy the code

If the table is already created, the get_table method can be used to assign the table to variables:

Hbase (main):011 > create 't',' F '0 row(s) in 1.2500 seconds => hbase ::Table -t hbase(main):012:0> TAB = get_table 't' 0 Row (s) in 0.0010 seconds => Hbase::Table -t Hbase (main):013:0> tab.put 'R1 ', 'f', 'V' 0 row(s) in 0.0100 seconds hbase(main):014:0> tab.scan row COLUMN+CELL R1 COLUMN = F :, timestamp=1378473876949, Value =v 1 row(s) in 0.0240 seconds hbase(main):015:0>Copy the code

The list feature has also been extended so that it returns a list of table names as a string. You can then use JRuby to script table manipulation based on these names. The list_snapshots command works similarly.

Hbase (main):016 > tables = list('ns:t.*') TABLE t 1 row(s) in 0.1040 seconds => #<#<Class: 0x7677CE29 >: 0x21D377A4 > hbase(main):017:0> tables.map { |t| disable t ; Drop T} 0 row(s) in 2.2510 seconds => [nil] hbase(main):018:0>Copy the code

10. Enable the debug mode

You can set the debugger in the shell to see more output when you run the command, such as more exception stack traces:

hbase(main):007:0> debug
Debug mode is ON

hbase(main):008:0> debug
Debug mode is OFF
Copy the code

Method 2:

$ ./bin/hbase shell -dCopy the code

Reprint please indicate the source! Welcome to follow my wechat official account [HBase Work Notes]