Recently, I have learned some HBase. All kinds of blogs that can be found on the Internet are quite old, and I installed the 3.0+ version, so I made a lot of mistakes when I first contacted them. I hereby record them so as not to get mentally retarded again, and also to have a further understanding of their features.

Written in fragments… If you have too many problems, you can integrate them

Novices like me can use my problem set as a reference for some troubleshooting ideas.

Because hbase3.0 version of the related resources is really a little less. Better to read an honest and reliable book, or read official documents. . – –

1. Design of ROWKEY and TIMESTAMP in re-centralization

Because ROWKEY and TIMESTAMP of Hbase are sorted based on lexicographical order, characters are matched based on this order in SCAN. Therefore, the design of ROWKEY and TIMESTAMP is a top priority. Reasonable design of ROWKEY can greatly reduce the speed of table response in various operations.

For starters with a small amount of data, if there is no clear design idea, rowKeys can be sorted in order, but remember to fill in zeros.

That is:

In order: [1 , 2 , 3.. N] The digits to be completed are: ['000001'.'000002'.'000003'. n]Copy the code

This aligns the number of digits so that the table is in order, otherwise your scan results will be unreliable.

For example, if you do not complete, you want to output the range STARTROW => ‘1’,STOPROW => ‘100’.

It’s likely to have only three rows: ‘1’, ’10’, ‘100’

Of course, the best case scenario is to take advantage of this feature in designing RowKeys and timestamps.

2. The VERSIONS feature

After you create the table initially, VERSIONS default to 1, meaning that when you insert data at different times, the table only keeps the latest one. So when we need to record different versions, we can use:

alter 'tablename', NAME => 'COLUMNFAMILY_name',VERSIONS => MAX_VERSION
Copy the code

This allows you to extend the corresponding column family. Different versions are sorted by TIMESTAMP, of course, as well as lexicographical order

We can use the following command to check

describe 'tablename'
Copy the code

3. Unreasonable Chinese character support of Hbase Shell

When you store a Chinese string, the hexadecimal encoding is displayed in Hbase Shell.

You can add:

scan 'tablename',FORMATTER => 'toString'
Copy the code

Convert its output to a string for display. This will display Chinese in the SHELL.

In addition, inputting Chinese characters in SHELL and modifying commands with Chinese characters are also extremely painful. It is recommended to write the command in any line of text first, and then copy it over. Don’t ask why, you can try this SHELL for yourself that causes brain hemorrhage.

About String Matching

All Hbase values are stored in Byte format. Therefore, comparators and comparison operators are required to filter matching strings.

Take the binary comparator, binary, and the comparison operator =.

To match the String String in your SHELL, you should say:

scan 'tablename', {FILTER => "SingleColumnValueFilter('ColFam','ColName',>=,'binary:String')"}
Copy the code

scan ‘tablename’,{FILTER => “SingleColumnValueFilter(‘ColFam’,’ColName’, = ,’ binary :String’)”}

An error is reported without adding any comparators. –

In JAVA, you would write:

BinaryPrefixComparator comp =new BinaryPrefixComparator(Bytes.toBytes("YourString"));

SingleColumnValueFilter filter = new SingleColumnValueFilter(
					Bytes.toBytes(cf),
					Bytes.toBytes(col),
					CompareOperator.EQUAL,
					comp
					);
Copy the code

It’s worth noting that the = symbol in the SHELL, the comparison operator, is called using the CompareOperator. ComOp prior to 3.0 is no longer recommended.

4. The problem we haven’t figured out yet

1. Hbase takes up a large amount of memory after running for a long time. However, I always shut down the table after each service.

2. Various pits of scan

Harm, honest whole book go -. –

This blog will be updated continuously… The problem that encounters can record down commonly, also welcome big guy to give suggestion, thank ~